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Abstract 

Scientist/engineer  productivity  in  a selected  Air  Force 
RAD  laboratory  was  measured  with  the  number  of  eight  differ- 
ent types  of  output  authored,  prepared  and/or  presented,  as 
reported  by  135  scientists/engineers. 

A weighting  scheme  was  developed  from  the  results  of  a 
questionnaire  administered  to  53  first  level  supervisors  in 
the  selected  laboratory.  These  weights  were  applied  to  the 
eight  output  types,  providing  a weighted  sum  measure  of  pro- 
ductivity. This  weighted  sum  was  compared  to  the  raw  sum  of 
the  eight  output  types.  The  correlation  between  raw  and 
weighted  output  was  found  to  be,  r=.985,  indicating  that  the 
application  of  the  weighting  scheme  only  slightly  changed  the 
rank  ordering  of  the  productivity  of  the  135  scientists/en- 
gineers . 

Through  zero  order  and  canonical  correlation  analyses, 
these  productivity  measures  were  compared  to  measures  of  23 

l 

organizational  and  individual  variables  present  in  the  lab- 
oratory. No  single  organizational  or  individual  variable 
was  shown  to  be  a correlate  of  all  eight  output  types.  How- 
ever, seven  variables  were  significant  zero  order  correlates 
of  weighted  output  and  had  substantial  canonical  correlation 
loadings  on  joint  criteria  that  included  weighted  output. 

Educational  level  and  frequency  of  attendance  at  pro- 
fessional society  meetings  were  shown  to  be  significant  cor- 
relates of  performance,  as  were  variables  concerning  frequen- 
cy of  communication,  both  within  a scientist/engineer’s  func- 

viii 


tional  section  and  outside  it. 

The  degree  to  which  a scientist/engineer  participates 
in  project  and  work  goals  was  found  to  be  a significant  posi- 
tive correlate  of  performance , as  was  the  degree  to  which  a 
scientist/engineer  perceives  he  is  rewarded  for  quantity  of 
output.  The  length  of  time  a scientist/engineer  has  under 
the  same  supervisor  was  found  to  be  negatively  correlated  to 
performance . 

Variables  found  to  have  little  or  no  association  with 
performance  were  age,  perceived  work  autonomy  and  perceived 
pressures  for  quality/quantity  of  output,  for  meeting  sched- 
ules, and  for  relevancy  of  work  to  the  Air  Force  mission. 

The  pattern  of  some  of  the  zero  order  correlations  be- 
tween the  organizational/individual  variables  and  the  indi- 
vidual output  types  indicated  the  possible  existence  of  broad 
based  groupings  of  the  eight  output  types. 
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QUANTITATIVE  SCIENTIST/ENGINEER  PRODUCTIVITY  AND  SONE 
ASSOCIATED  INDIVIDUAL  AND  ORGANIZATIONAL  VARIABLES 

I . Introduction 

Background 

It  is  perhaps  an  overstatement  to  submit  that  all  pro- 
fessional Research  and  Development  (RftD)  managers  are  aware 
of  the  impact  of  various  organizational  or  situational  vari- 
ables on  the  performance  of  the  scientists  and  engineers 
whom  they  manage . However , some  managers  are  not  only  a- 
ware  of  this  impact,  but  are  actively  engaged  in  programs 
to  improve  the  performance  of  scientists/engineers  under 
them.  In  1975»  Bender  (Ref  7*35)  reported  the  results  of 
a survey  of  Industrial  Research  Institute  member  companies 
which  were  asked  if  they  "did  (do)  have  programs  to  stimu- 
late creativity/productivity?"  Bender  did  not  report  the  to- 
tal number  of  respondents  to  the  survey,  however  nineteen 
companies  answered  the  question  affirmatively,  identifying 
44  different  programs.  This  seems  to  indicate  the  effort 
that  managers  are  taking  to  provide  an  organizational  envi- 
ronment conducive  to  research  productivity.  The  fact  that  a 
large  variety  of  programs  have  been  implemented  may  also 
point  out  that  a "best"  method  to  stimulate  productivity  does 
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not  exist,  but  varies  instead  from  one  organization  to  the 
next,  partially  based  on  which  factors  or  organizational 
variables  are  controllable  by  the  manager. 
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In  the  past  fifteen  years,  there  have  been  a number  of 
empirical  studies  conducted,  attempting  to  determine  the  or- 
ganizational factors  or  variables  that  impact  the  productiv- 
ity of  the  individual  scientist/engineer.  Martino  (1973)  re- 
ported 87  such  empirical  studies  and  postulations  in  which 
13  various  measures  of  scientific  performance  and  53  associ- 
ated individual  and  organizational  characteristics  were  in- 
vestigated (Ref  12«72).  Many  of  these  studies  investigated 
similar  variables  and  related  them  to  similar  measures  of 
scientific  performance  ir  different  organizations.  Chapter 
2 will  point  out  several  different  techniques  that  have  been 
employed  to  determine  scientific  performance.  Additionally, 
there  is  no  consistently  defined  term  for  scientific  per- 
formance. Depending  on  the  investigator,  terms  such  as  cre- 
ativity, innovation,  productivity  and  output  have  all  been 
used. 

Stahl  (1975)  confronted  these  many  dissimilarities  when 
he  prepared  an  extensive  literature  survey  in  his  investiga- 
tion to  determine  a set  of  organizational  variables  that  was 
associated  with  innovation  and  productivity  of  scientists 
and  engineers  (Ref  22).  His  survey  included  most  of  the  87 
previously  referenced  studies  along  with  numerous  others. 
Chapter  2 will  provide  greater  detail  concerning  Stahl's  in- 
vestigation. However,  in  general,  he  used  a peer  rating 
technique  to  determine  measures  of  innovation  and  productiv- 
ity. These  measures  were  compared  to  measures  of  organiza- 
tional and  individual  variables  obtained  through  the  use  of 
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a questionnaire  completed  by  the  scientists  and  engineers  in 
three  Air  Force  R&D  laboratories. 

Rather  than  using  a peer  or  supervisor  rating  of  product- 
ivity, the  issue  to  be  explored  in  this  effort  is,  what  are 
the  relationships  between  an  objective , quantifiable  measure 
of  scientist/engineer  productivity  and  certain  selected  or- 
ganizational and  individual  variables? 

The  Problem  and  Its  Limitations 

Specifically,  what  are  the  relationships  between  certain 
organizational  variables  and  output  of  working  level  scien- 
tists/engineers in  an  Air  Force  R&D  laboratory?  Do  these 
relationships  change  when  a weighting  scheme  is  applied  to 
the  multiple  dimensions  of  output?  How  do  these  relation- 
ships compare  to  those  developed  from  a peer  rated  measure 
of  productivity? 

r 

The  questions  posed  above  indicate  the  limitations 
within  which  this  research  will  be  accomplished.  First  note 
that  this  research  effort  is  not  concerned  with  an  investi- 
gation to  determine  an  all-inclusive  set  of  organizational 
variables  to  be  tested,  or  even  a "best"  set.  It  does  in- 
clude the  selection  of  variables  that,  in  previous  studies, 
have  been  shown  to  be  correlated  to  some  definition  of  sci- 
entific and  engineering  performance.  For  purposes  of  com- 
parison, the  set  of  variables  utilized  is  similar  to  the  set 
investigated  in  Stahl's  effort. 

Second,  the  dimensions  of  scientific  performance  must 
fit  the  objective,  countable  criterion.  That  is  to  say,  the 
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somewhat  subjective  notions  of  creativity  and  innovation  are 
not  being  measured  except  to  the  extent  that  they  are  reflect- 
ed in  the  tangible  output  of  the  scientist/engineer.  Strict 
adherence  to  this  criterion  of  scientific  performance  does 
have  an  inherent  limitation,  centered  on  the  notion  of  non- 
equality of  content  of  the  various  types  of  tangible  scien- 
tific and  engineering  output. 

Previous  empirical  studies  have  attempted  to  minimize 
the  effects  of  this  non-equal  content  aspect  throu^i  the  ap- 
plication of  a weighting  scheme  in  the  form  of  subjective 
ratings  of  the  individual  scientist/engineer . Any  set  of 
subjective  weights  is  limited  by  the  accuracy  and  reliability 
of  the  method  used  to  determine  the  weight. 

Finally,  there  is  a limitation  on  the  sample  size  of 
scientists/engineers  to  be  tested  due  to  time  and  cost  con- 
straints. This  effort  is  limited  to  the  data  gathered  from 
a single  Air  Force  R&D  laboratory. 

Research  Objectives 

1 . Modify  the  questionnaire  developed  by  Stahl  to  mea- 
sure similar  R&D  organizational  variables. 

2.  Develop  a questionnaire  for  use  in  determining  a 
weighting  scheme  to  be  applied  to  output  dimensions. 

3.  Develop  a questionnaire  enabling  respondents  to  re- 
port their  objective  output. 

4.  Determine  the  relationships  between  organizational 
variables  and  both  weighted  and  unweighted  output. 
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Chapter  II . Literature  Review 


Introduction 

A review  of  the  pertinent  literature  was  required  to 
familiarize  the  researcher  with  the  previous  studies  that 

have  been  conducted.  This  familiarization  was  required  to 

/ 1 

gain  an  understanding  of  (1)  the  aspect  of  scientist/engin- 
eer productivity  investigated,  (2)  the  various  individual 
and  organizational  variables  investigated,  (3)  the  measure- 
ment techniques  employed  and  (4)  the  findings  reported. 

As  noted  in  Chapter  1 , a recent  literature  review  was 
conducted  by  Stahl  in  1975.  identifying  the  major  studies 
conducted  during  the  previous  fifteen  years.  Consequently, 
what  follows  in  this  chapter  is,  for  the  most  part,  a sum- 
mary of  the  studies,  articles  and  postulations  that  were  re 
viewed  by  Stahl.  Also  described  however,  are  some  studies 
not  included  in  Stahl’s  review. 


Scientist/fcngineer  Productivity  Measures 

Vincent  and  Mirakhor  state , 

"It  is  important  to  note  at  the  outset  that 
the  study  deals  with  an  area  of  research  in 
which,  despite  overwhelming  volume  of  empir- 
ical studies,  there  is  considerable  confus- 
ion not  the  least  of  which  confcems  the  ter- 
minology used.  The  difficulty  generally 
stems  from  the  fact  that  there  is  little  a- 
greement  among  the  researchers  on  the  defin- 
ition of  terms  used."  (Ref  24j45) 

The  confusion  alluded  to  is  certainly  evidenced  by  the 

various  measures  of  scientific  and  engineering  productivity 

used  in  empirical  studies.  Terms  such  as  performance,  inno 


vation,  creativity  and  productivity  have  all  been  used  in 
different  studies,  referring  to  that  which  is  representa- 
tive of  the  output  of  a scientist/engineer. 
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Given  the  quantitative  thrust  of  this  research,  the 
question  became,  what  are  the  objective  measures  of  scien- 
tist/engineering productivity? 

As  noted  by  Stahl  (Ref  22«85),  Pelz  and  Andrews  inclu- 
ded in  the  list  of  scientific  output 

"...(a)  patents  or  patent  applications, 

(b)  published  papers,  (c)  books  and  (d) 
unpublished  technical  manuscripts,  re- 
ports or  formal  talks." 

Mullins  (Ref  15:52)  used  the  number  of  publications  re- 
gardless of  content  area  along  with  individual  supervisor 
ratings  as  a measure  of  productivity  and  Meltzer  and  Salter 
(Ref  14:351)  asked  respondents  to  report  the  "number  of  re- 
search papers  published  in  the  last  three  years." 

In  Eiduson’s  study  on  change  in  productivity  rate,  the 
index  of  productivity  was  the  number  of  scientific  publica- 
tions (Ref  7 i 57). 

Hill  used  the  publication  rate  as  his  measure  of  pro- 
ductivity because  the  "assessment  of  publication  rate  was  a 
key  indicator  of  scientific  productivity  to  more  senior  man- 
agement" (Ref  10:13). 

In  conjunction  with  a subjective  rating,  Parris  used  a 
self  report  by  the  scientists/engineers  on  number  of  patents/ 
patent  applications,  number  of  unpublished  technical  reports, 
and  number  of  formal  talks,  as  his  measure  of  productivity 
(Ref  8:9).  Andrews  and  Farris  also  used  a peer  rating  in 
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conjunction  with  a self  report  on  number  of  technical  re- 
ports authored  (Ref  4«497). 

As  is  somewhat  apparent  from  the  above,  many  research- 
ers recognized  the  problem  of  non-equal  content  when  using 
a strictly  quantitative  output  measure  of  productivity. 

That  is  to  say,  that  many  researchers  were  concerned  that 
quality,  value,  importance  or  other  content  factors  were  dif' 
ferent  from  one  output  type  to  another.  Consequently,  sub- 
jective aspects  of  scientist/engineer  output,  such  as  cre- 
ativity and  innovation  were  measured  through  the  use  of  su- 
pervisor or  peer  ratings.  These  subjective  ratings  of  indi- 
vidual productivity  were  then  combined  with  reported  output 
to  obtain  a composite  productivity  measure. 

The  peer  rating  utilized  by  Andrews  and  Parris  (Ref  4« 
497)  requested  the  individual’s  peers  to  rate  him  on  pro- 
ductivity, innovation,  contribution  to  science  and  useful- 
ness of  products.  Farris  (Ref  8«9-l6)  asked  the  individual 
to  provide  a rating  on  contribution  to  the  scientific  disci- 
pline and  usefulness  to  the  organization. 

Pelz  (Ref  16»310)  also  used  a peer  rating,  averaging 
the  ratings  of  as  many  as  14  peers  per  individual  to  obtain 
a performance  measure . Andrews  also  used  a peer  rating  on 
contribution  and  usefulness  to  determine  a productivity  mea- 
sure (Ref  3 1 1 64 ) . 

The  above  cited  studies  are  examples  of  commonly  used 
methods  for  measuring  scientist  and  engineer  productivity. 
Those  being  either  strict  quantitative  measures,  or  quanti- 


tative  measure b used  with  subjective  peer  or  supervisor  rat- 
ings of  contribution  and  usefulness.  In  addition  to  these 
techniques,  Stahl's  review  of  literature  (Ref  22«34-66)  in- 
cludes studies  that  have  one  or  more  of  the  following  types 
of  productivity  measures! 

1.  Self  rating  on  creativity/innovation, 

2.  Supervisor  or  peer  ratings  of  communication 
skill  and  skill  with  people, 

3.  Additional  quantitative  measures  such  as  a- 
wards  received,  promotion  rate,  percent  of  projects  on  sched- 
ule and  percent  of  projects  within  cost,  and/or 

4.  Scores  on  specific  creativity  tests  such  as 
Remotes  Associates  Test  and  Guilford  Tests. 

It  is  interesting  to  note  that  none  of  the  studies  spe- 
cifically cited  above,  or  the  additional  measurement  techn- 
iques listed,  apply  a weight  to  the  output  itself,  but  ra- 
ther they  apply  a weight  to  the  individual  scientist/engi- 
neer. In  only  one  instance  did  the  researcher  find  a study 
that  applied  weights  to  specific  quantitative  dimensions  of 
output.  Vincent  and  Mirakhor  (Ref  24i50)  used  technical  pa- 
pers published,  patent  disclosures  or  inventions,  and  presen- 
tations made  as  a measure  of  productivity.  They  noted  that, 

"However  it  is  not  considered  that 
these  three  indicators  are  of  equal 
difficulty  or  importance.  For  man- 
agement dec isi on-making  purposes, 
the  following  weighting  was  estab- 
lished within  the  organization!  5 
on  patents,  3 on  papers  and  1 on 
presentations . " 

Vincent  and  Mirakhor  applied  these  weights  to  the  output  as 
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reported  by  the  individual  scientists  and  engineers  to  est- 
ablish a weighted  output  measure.  Not  included  in  their 
findings  however,  was  an  indication  of  the  degree  of  corre- 
lation between  the  weighted  and  unweighted  measures.  As 
this  research  is  concerned  with  similar  weighted  and  un- 
weighted measures,  this  information  would  have  been  of  value 
for  comparison  purposes. 

Table  I below  presents  a summary  of  the  productivity 
measures  described  in  this  section. 


Table  I.  Productivity  Measures 


A.  Quantitative  Measures 


1 . Patent/patent  applica- 
tion 

2.  Published  reports 

3.  Unpublished  manuscripts 

4.  Journal  articles 

5.  Books 

6 . Formal  talks 

7.  Productivity  rate 

8.  Inventions 

9 . Awards  received 

10.  Promotion  rate 

11.  Percent  project  is  on 
schedule/within  cost 


B.  Qualitative  Measures 


C . '-'sightings 


1 . Supervisor/peer/self 
ratings  on« 

a.  Contribution  to  sci- 
entific discipline 

b.  Usefulness  of  pro- 
ducts 

c . Innovation 

d.  Productiveness 

e.  Creativity 

2.  Creativity  tests  1 

a.  Remote  Associates 

b.  Guilford 

1 . Based  on  individual 

2.  Based  on  output 


In  sum,  there  is  a vast  number  of  empirical  studies  and 
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postulations  concerning  various  measures  of  scientist/engi- 
neer productivity,  ranging  from  pure  objective  counts  of 
output  to  purely  subjective  peer  rating  measures.  Interest- 
ly,  however  is  the  fact  that  only  one  of  the  studies  noted 
used  a weighting  scheme  applied  to  output  dimensions.  Con- 
sequently, one  of  the  objectives  of  this  research  is  to  de- 
velop and  employ  a weighting  scheme  that  is  applied  to  a 
list  of  quantitative  output  dimensions. 

Organizational/Individual  Variables 

Identification.  The  list  of  individual,  group  and  or- 
ganizational variables  that  have  been  investigated  in  pre- 
vious studies  is  quite  extensive.  Martino  (1973)  divided 
53  variables  into  eight  separate  categories  (Ref  12«69-70). 
These  categories,  with  some  examples  of  variables  in  each, 
are  listed  in  Table  II. 


Table  II . Martino ' s Variables 


Examples' 


Variable  Category  

Controllable  Variables 
(Climate) 


Controllable  Variables 
(Physical) 

Uncontrollable  Labora- 
tory Variables 


Personal  Characteristics 
(Inherent) 

Personal  Characteristics 
(Acquired) 


project  organization, 
leadership  style,  di- 
versity of  work 

equipment,  funding, 
group  size 

time  in  organization, 
project  phase,  type  of 
activity 

psychological  test 
scores  (Guilford,  RAT) 

prior  employment,  ac- 
ademic major,  degree 
level 
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Table  II.  - continued 
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Variable  Category 


Examples 


Personal  Characteristics 
(Descriptive) 

Intermediate  Variables 


Researcher  Activities 


age,  time  since  de- 
gree 

job  satisfaction, 
awareness  of  man- 
agement desires 

consultation  with 
others  in  the  labor- 
atory, attendance  at 
professional  meetings 


Similarly,  Stahl  categorized  117  variables  that  were 
"statistically  significant  or  substantial"  into  four  general 
categories.  These  categories,  with  some  examples  of  vari- 
ables in  each,  are  included  in  Table  III. 


Table  III.  Stahl's  Variables 

Variable  Category  Examples 

Group  Variables  group  heterogeneity, 

frequency  of  commun- 
ication, group  age, 
size 

Supervisor  Variables  frequency  of  contact, 

participation  in  pro- 
ject selection,  re- 
ward structure 


Fi 

6 '1 


Other  Variables  salary,  diversity 

of  activities,  lab- 
oratory budget 

Statistical  Control  length  of  experience, 

Variables  type  of  work,  time 

in  laboratory  divi- 
sion 


Tables  II  and  III  provide  an  indication  of  the  diver- 
sity of  the  kinds  of  variables  that  have  been  investigated 
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in  previous  studies.  It  was  from  this  list  of  variables 
that  a set  to  be  investigated  in  this  research  was  selected. 
Chapter  3 presents  more  on  the  selection  of  the  variables 


that  were  investigated. 

Correlates  of  Scientist/Engineer  Productivil 


As  an 


objective  of  this  research  is  to  provide  a basis  for  compar- 
ison to  Stahl' 8 effort,  the  identification  of  the  organiza- 
tional and  individual  variables  that  he  showed  to  be  signifi- 
cant correlates  of  peer  rated  measures  of  productivity  and 
innovation  is  in  order. 


Variables  that  Stahl  showed  as  being  negative  correlates 
of  both  innovation  and  productivity  were  age,  length  of  time 
as  a scientist/engineer,  and  length  of  Federal  Government 
employment.  Positive  correlates  of  both  criteria  were  shown 
to  be  education  and  frequency  of  communication  with  other 
scientists/engineers  in  the  same  group.  Additionally,  the 
perceptual  variables  of  pressure  concerning  the  relevancy  of 
work  to  the  Air  Force  mission,  participation  on  project  se- 
lection, empathy  and  rewards  for  innovation  were  all  signif- 
icant positive  correlates  of  productivity  (Ref  22«152-153). 
Stahl  noted  that  these  findings  were  supported  by  the  re- 
sults of  numerous  other  researchers  (Ref  22  j155) • 

Vincent  and  Mirakhor  failed  to  mention  the  degree  of 
correlation  between  weighted  and  raw  output.  However,  they 
did  find  age,  job  satisfaction,  and  ability  (education)  to 
be  correlates  of  a weighted  productivity  measure,  similar 
to  the  one  used  in  this  research  (Ref  24i50). 
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Hill  (1970)  reported  that  a change  in  leadership  style 
from  permissive/democratic  to  autocratic,  caused  a signifi- 
cant reduction  in  the  publication  rate  of  not  only  the  super 
visor's  immediate  subordinates,  but  also  in  the  publication 
rate  of  scientists  in  an  associated  section  (Ref  10:10-20). 
Pelz  found  that  "high  contact  (more  than  several  times  week- 
ly)" between  scientists  and  "colleagues  who  are  on  the  aver- 
age dissimilar"  was  an  individual  variable  associated  with 
high  performance  (Ref  16:314-315). 

Meltzer  and  Salter  found  that  laboratory  variables  such 
as  funds  available  and  facilities,  and  individual  variables 
such  as  ability  and  job  freedon  were  correlates  of  product- 
ivity (Ref  14:351-362).  Farris  reported  that  the  variables, 
highest  degree  earned,  length  of  time  since  degree  and 
length  of  time  with  laboratory  accounted  for  80  percent  of 
the  variance  in  scientist/engineer  performance  (Ref  8:9-16). 

In  relating  productivity  rate  to  age,  Eiduson  found 
that  productivity  rate  remained  constant  -or  increased  up  to 
the  age  of  60  (Ref  7:57-63).  Andrews  found  that  education, 
length  of  work  experience,  type  of  research  setting  and  time 
between  earning  a B.S.  and  PhD  were  all  correlates  of  pro- 
ductivity (Ref  3:185). 

To  summarize,  there  is  a preponderance  of  organization- 
al, group  and  individual  variables  from  which  to  choose, 
that  have  been  shown  to  be  significant  correlates  of  scien- 
tist/engineer productivity.  An  investigation  of  the  major- 
ity of  these  variables  to  determine  a "best  set"  is  outside 


the  scope  of  this  research.  However,  from  this  list  a set 
will  be  selected  for  investigation  of  its  association  with 
a quantitative  measure  of  productivity. 


Ill . Methodology 


Introduction 

What  follows  is  a description  of  the  methodology  employ* 
ed  during  the  conduct  of  this  research  effort.  Included  are 
sections  concerning  (1)  the  measurement  techniques  employed, 

(2)  the  selection  and  characteristics  of  the  subject  sample, 
and  (3)  the  analytic  methods  utilized.  Where  appropriate, 
each  of  these  sections  includes  justifications  of  the  action 
taken  and  the  methodology  employed. 

Application  of  the  Questionnaire 

As  can  be  gathered  from  a review  of  the  empirical  stud- 
ies noted  in  Chapter  2,  the  use  of  a questionnaire  as  a data 
collection  technique  is  commonplace  in  this  area  of  manage- 
ment research.  The  use  of  the  questionnaire  has  the  advan- 
tage of  providing  a large  volume  of  data  in  a relatively 
short  period  of  time  when  compared  to  an  alternative  method 
such  as  personal  interviews.  This  aspect  alone  made  it  part- 
icularly favorable  in  light  of  the  time  and  money  constraints 
under  which  this  research  was  conducted.  Additionally,  the 
questionnaire  used  by  Stahl  was  in  existence  and  pretested. 

With  only  a few  modifications,  that  questionnaire  was  ade- 
quate to  obtain  part  of  the  data  required  for  this  research. 

The  survey  instrument  used  in  this  effort  was  composed 
of  two  separate  questionnaires.  Part  I was  sent  to  first  lev- 
el supervisors  and  was  designed  to  provide  a weighting  scheme 
to  be  applied  to  the  various  dimensions  of  scientist  and  engin- 
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eer  output.  Part  II  was  sent  to  working  level  scientists 
and  engineers  and  provided  a measurement  for  the  organiza- 
tional variables  in  question  and  the  quantitative  output  of 
the  individual  scientist/engineer. 

Output  Measurement 

Criteria ■ The  overriding  criterion  used  in  the  deter- 
mination of  a list  of  output  to  be  investigated  was  that  it 
be  objective.  That  is  only  scientist/engineer  output  that 
was  a countable,  tangible  product  was  included. 

A preliminary  list  was  compiled  from  a review  of  the 
empirical  stuiies  cited  in  Chapter  2 and  from  the  research- 
er's own  experience.  This  list  included  (1)  test  reports, 

(2)  test  plans,  (3)  journal  articles,  (4)  patent/patent  ap- 
plications, (5)  contract  statements  of  work,  (6)  hardware/ 
software  specifications,  and  (7)  oral  presentations. 

This  list  was  later  modified  as  explained  in  a follow- 
ing section,  however  an  additional  criterion  for  inclusion 
is  perhaps  obvious  , even  from  the  preliminary  list.  Only 
those  types  of  output  that  are  considered  non-routine  and 
non-administrative  in  nature  were  included  in  the  list. 

As  can  be  gathered  from  the  final  list,  this  criterion  did 
not  eliminate  types  of  output  such  as  management  presentations, 
which  might  be  more  communicative  in  nature  than  more  clas- 
sical scientist/engineer  output  such  as  technical  reports  and 
journal  articles,  which  involve  a more  investigative  and  ex- 
perimental orientation.  Nor  were  output  dimensions  such  as 
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procurement  package  preparations  and  evaluations  of  propos- 
als eliminated,  both  of  which  are  quite  representative  of 
an  engineer's  proiMctivity  in  an  Air  Force  R&D  laboratory. 

Items  such  as  weekly  progress  reports,  daily  consulta- 
tions with  peers,  and  other  routine  type  activities  were  not 
deemed  appropriate  measured  of  productivity.  Also  elimin- 
ated were  items  such  as  effectiveness  ratings  (OER's)  pre- 
parations and  public  relations  activities  which,  although 
non-routine,  cannot  be  considered  output  of  a scientist/ 
engineer. 

Additionally,  certain  types  of  output,  which  may  be 
considered  representative,  did  not  fit  the  objective,  count- 
able criterion.  Items  such  as  coordination  with  other  gov- 
ernment agencies,  although  perhaps  representing  a major 
amount  of  a scientist/engineer's  workload,  did  not  lend  it- 
self to  quantification. 

In  sum  then,  the  primary  considerations  for  inclusion 
in  the  list  were  the  objective,  countable  criterion  and  the 
non-routine,  non-administrative  aspect  of  the  types  of  output 
that  indeed  were  countable.  With  these  considerations,  in 
mind  the  final  list  of  output  dimensions  was  established. 

Identification/Measurement . During  a brief  meeting  in 
June  with  representatives  of  the  Air  Force  laboratory  to  be 
investigated,  the  preliminary  list  of  output  dimensions,  cited 
in  the  previous  section,  was  discussed.  These  representa- 
tives expressed  a desire  to  further  review  the  list  with 
the  objective  of  modifying  it  to  include  output  dimensions 
that  more  closely  resembled  the  output  types  of  the  partic- 
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ular  laboratory.  A subsequent  trip  to  the  laboratory  was  re- 
commended . 

In  July  the  researcher  personally  visited  the  facilities 
at  the  laboratory  and  part  of  the  two-day  trip  was  concerned 
with  finalizing  the  list  of  output  dimensions.  It  became 
apparent  that  a major  consideration  in  finalizing  the  list 
of  output  dimensions  was  to  ensure  that  it  included  items 
that  were  readily  identifiable  to  those  who  were  reporting 
their  output  and  to  those  who  were  to  apply  weights.  With 
this  additional  consideration  in  mind,  the  final  list  of  out- 
put included  (1)  technicax  reports,  (2)  work  unit  planning 
documents,  (3)  journal  articles,  (4)  procurement  packages, 

(5)  evaluations  of  proposals,  (6)  computer  codes,  (7)  tech- 
nical presentations  and  (8)  management  presentations. 

From  the  preliminary  list,  test  reports  and  test  plans 
were  combined  into  the  technical  report  category.  The  justi- 
fication for  this  action  was  primarily  due  to  the  fact  that, 
in  the  particular  laboratory  in  question,  test  plans  and 
reports  were  generally  prepared  for  the  purpose  of  inclu- 
sion in  a full  technical  report.  It  was  felt  that  the  use 
of  technical  report,  in  lieu  of  test  re ports/ plans,  would 
insure  less  confusion  on  the  part  of  the  respondent  from  a 
reporting  stnadpont.  The  phrase,  "including  test  reports" 
was  added  for  further  clarification. 

The  preliminary  category  of  oral  presentations  was  di- 
vided into  technical  presentations  and  management  presenta- 
tions. T’his  again  was  done  primarily  to  improve  the  re- 
porting accuracy  of  the  respondent.  The  phrase  "includ- 
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ing  those  made  outside  the  laboratory”  was  added  to  emphasize 
the  point  that  routine  consultation  and  reporting  to  super- 
visors or  peers  was  not  to  be  reported.  The  phrase,  "bud- 
get reviews,  for  example"  was  added  to  provide  a differen- 
tiation between  the  technical  and  management  orientation 
of  the  presentation.  It  was  expected  that  this  would  en- 
hance the  ability  of  the  weigh ter s and  the  output  reporters 
to  accurately  respond. 

The  contract  statements  of  work  was  divided  into  pro- 
curement packages  and  evaluations  of  proposals.  This  divi- 
sion allowed  the  weighters  and  the  output  reporters  to  dif- 
ferentiate between  the  two  aspects,  preparation  and  evalu- 
ation, of  the  contract  statement  of  work  item. 

Neither  the  patent/patent  application  item  or  the  hard- 
ware/software specification  item  were  included  in  the  final 
list,  per  recommendation  of  the  laboratory  personnel  con- 
sulted. Concerning  patents,  they  noted  that  none  had  been 
applied  for  by  individuals  in  the  laboratory  for  several 
years  and  consequently  patents  did  not  really  represent  the 
output  of  the  scientists/engineers  in  that  laboratory.  It 
is  somewhat  understandable  that  patents  would  be  uncommon 
in  this  particular  laboratory,  which  deals  with  highly  soph- 
isticated and  national  security  sensitive  weapons  systems. 

Hardware/software  specifications,  although  a major 
type  of  output  in  defense  weapon  system  development,  again 
was  not  considered  representative.  There  were  scientists 
and  engineers  in  the  laboratory  that  reviewed  defense  con- 
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tractor- submitted  specifications , however  few  actually  en- 
gaged in  the  total  preparation  and  development  of  specifi- 
cations. 

The  journal  article  item  was  retained  in  the  final 
list.  However  the  phrase,  "either  published  or  submitted," 
was  added  to  further  enhance  reporting  accuracy  on  the  self- 
report  portion  of  the  questionnaire. 

Items  in  the  final  list  that  were  not  in  the  prelim- 
inary list  were,  work  unit  planning  documents  and  computer 
codes.  A work  unit  planning  document  is  a report  concern- 
ing a project  or  particular  portion  of  a project.  This  type 
of  document  would  include  information  concerning  scheduling, 
funding  and  manpower  requirements  for  the  project,  in  addi- 
tion to  identifying  required  external  agency  support.  A 
computer  code  is  essentially  a computer  program  developed 
to  meet  either  simulation  or  analytic  requirements  of  a 
particular  laboratory  project.  It  was  pointed  out  by  the 
laboratory  personnel  interviewed  that  these  items  did  in 
fact  represent  a measure  of  scientist/engineer  productivity 
that  met  both  the  non-routine , non-admini strati ve  criterion 
and  the  objective,  countable  criterion. 

A final  category  titled,  "other"  was  added  to  allow 
the  respondent  to  report  types  of  output  that  may  not  have 
been  specifically  identified. 

With  the  final  list  thus  established,  a questionnaire 
was  developed  asking  for  a self  report  on  the  numbers  of 
each  type  of  output  prepared,  presented,  authored  and/or  co- 
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authored  over  a given  period  of  time . The  time  period  cho- 
sen for  investigation  was  that  of  the  most  recent  two  years. 
The  two  year  period  was  chosen  for  two  primary  rea- 


sons. There  was  a desire  to  associate  the  output  reported 


with  the  organizational  variables  present  in  a given  func- 
tional section  or  group.  Consequently,  the  applicable  report- 
in  time  period  must  be  such  .that  it  generally  corresponded 
with  the  period  of  time  the  respondent  was  located  in  a giv- 
en section  or  group.  Due  to  the  turnover  rate  and  individ- 
ual scientist/engineer  mobility  within  the  laboratory,  it  was 
felt  that  only  a minority  of  prospective  respondents  would 
have  a length  of  time  in  a given  section  of  three  or  more 
years.  Using  a two  year  time  period,  a greater  number  of 
respondents  would  have  time  in  section  generally  correspond- 
ing to  the  output  reported.  It  was  also  felt  that  the  re- 
spondent could  more  accurately  recall  his  output  over  two 
years  rather  than  over  a three  year  period.  Additionally, 
it  was  felt  that  a period  greater  than  one  year  was  required 
to  ensure  that  the  respondent’s  output  was  a function  of  the 
organizational  variables  measured  in  his  present  section  and 
not  a function  of  a prior  set  of  organizational  variables. 

V’ith  the  list  and  time  period  thus  established,  the  re- 
spondent was  simply  asked  to  report  the  quantity  of  each  type 
of  output  over  the  two  year  period.  This  report  established 
the  raw  output  measure  of  scientist/engineer  productivity 
being  used  in  this  research  effort.  The  weighting  scheme, 
described  below,  was  then  applied  to  the  reported  raw  out- 
put and  a weighted  output  was  established  for  each  scien- 
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ti8t/engineer . 

Weighting 

Objective.  As  touched  on  in  Chapter  2,  a major  draw- 
back In  using  a strictly  tangible  output  measure  of  product- 
ivity centers  on  the  notion  of  non-equality  of  the  content 
of  various  dimensions  of  output.  That  is  to  say,  not  all 
types  of  output  can  be  considered  equal  in  importance  or 
value  . Also  as  noted  by  Stahl  (Ref  22 187-88)  the  raw  num- 
bers  of  publications  and  presentations  in  many  cases  is  a 
function  of  the  organizational  emphasis  and  mission.  Addi- 
tionally, in  the  Air  Force,  security  considerations  may  pre- 
sent further  limitations  to  the  productiveness  of  a scientist/ 
engineer  in  terms  of  patents,  journal  articles  and  other  i- 
tems  that  would  otherwise  be  subject  to  public  dissemination. 

By  employing  the  questionnaire  directed  at  the  first  lev- 
el supervisor,  an  attempt  was  made  to  reduce  the  effects  of 
this  non-equality  notion.  That  is,  an  attempt  was  made  to 
determine  a weighting  scheme  that  would  replace  the  equally 
weighted  raw  output  which  is  implicitly  used  with  a strict- 
ly quantitative  output  measure  of  productivity. 

Previous  studies  employing  a peer  rating  or  supervisor 
rating  of  individual  productivity,  innovation  and/or  crea- 
tivity, have  been  combining,  in  many  cases,  an  objective 
measure  with  a rating  of  the  individual’s  contribution  to 
science,  originality  of  work,  usefulness  of  products  or  some 
other  subjective  aspect  of  scientist/engineer  productivity. 

In  effect,  this  type  of  productivity  measure  involves  the 
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application  of  a weight,  not  to  the  type  of  output,  but  to 
the  individual  himself.  There  are  certain  drawbacks  to  the 
individual  rating  technique.  Stahl  (Ref  22«18?)  points  out 
the  need  to  have  4 to  5 raters  per  ratee  to  obtain  a higher 
reliability  rating.  This  may  not  be  possible  due  to  the 
limited  number  of  peers  or  raters  that  are  familiar  enough 
with  the  individual  in  question  to  provide  a fair  rating. 
Additionally,  Stahl  notes  the  difficulties  of  using  a peer 
rating,  that  are  encountered  by  a researcher  who  requires  a 
reasonable  response  rate  from  a questionnaire . 

These  two  issues,  along  with  a desire  to  place  a weight 
on  the  type  of  output,  rather  than  on  the  individual  himself, 
were  the  reasons  for  discarding  the  peer  rating  technique. 

The  questions  then  become,  who  should  be  asked  to  apply  the 
weight  to  the  set  of  output  dimensions,  and  what  should  the 
weighting  criteria  be? 

Criteria.  In  order  to  determine  who  should  apply  the 
weights,  the  criteria  for  weight  determination  must  first  be 
established. 

It  is  important  to  understand  that  the  researcher  was 
desirous  to  have  an  output  weighting  scheme  for  the  labora- 
tory as  a whole,  rather  than  a multiple  set  of  weights  to 
be  applied  according  to  division  or  branch  within  the  lab- 
oratory . 

The  whole  notion  of  productivity  used  in  this  research 
effort  is  output  oriented.  This  means  that  regardless  of 
inputs  such  as  time,  money,  and  effort  expended  by  the  sci- 
entist/engineer, the  productivity  measure  was  strictly  his 
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output,  i.e.,  what  he  has  to  show  for  his  effort 
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With  these  understandings,  it  was  incongruous  to  ask 
the  weighters  to  base  their  weights  on  the  notion  of  the  a- 
mount  of  input  required  to  produce  the  output.  Another  cri- 
terion must  be  used. 

The  criteria  finally  decided  upon  were  based  on  the  i- 
dea  of  value  or  importance  to  the  laboratory  as  a whole. 

Each  weighter  was  asked  to  apply  a weight  that  he  felt  best 
represented  the  "relative  importance"  of  that  type  of  output 
to  the  laboratory  in  question.  This  criterion  best  fit  the 
conceptual  framework  of  the  researcher  in  that  the  weights 
were  not  being  based  on  the  input  required  to  produce  the 
output  dimension  being  weighted.  Also,  the  weighters  were 
required  to  view  the  output  in  its  importance  to  the  labor- 
atory as  a whole  and  not  just  a particular  (their  own)  por- 
tion of  it. 

With  the  criteria  established,  the  selection  of  the 
weighters  could  then  be  made. 

Selection  of  First  Level  Supervisor.  Given  that  weights 
were  to  be  applied  on  the  basis  of  relative  importance  to 
the  laboratory  in  general,  it  was  necessary  to  choose  a set 
of  weighters  who  (1)  had  a feel  for  the  value  of  each  type 
of  output  and  (2)  did  not  totally  base  their  weights  on  the 
particular  output  produced  in  their  own  section  or  group. 
Additionally,  the  set  of  people  to  apply  weights  must  com- 
prise a sample  large  enough  to  provide  a reliable  measure 
from  a statistical  standpoint. 


carded  were  the  working  level  scientists  and  engineers.  Al- 
though this  group  offered  the  largest  sample  from  which  to 
choose,  the  researcher  felt  that  the  applied  weights  would 
he  overly  biased,  favoring  the  type  of  output  with  which 
the  working  level  scientists/engineers  were  most  familiar. 

It  was  felt  that  this  bias  would  be  furthered  compounded  due 
to  the  fact  that  these  same  persons  would  be  completing  the 
self  report  of  output.  It  seemed  natural  that  they  would 
weight  those  dimensions  reported  more  heavily. 

Another  group  of  possible  weighters  were  individuals  at 
a higher  level  of  supervision  such  as  division  and  branch 
chief 8 which  are  somewhat  removed  from  the  working  level. 

This  group  would  not  suffer  from  the  same  bias  as  that  of 
the  working  level  group  and  would  probably  have  a better 
feel  for  the  importance  of  certain  types  of  output  to  the 
laboratory  as  a whole.  However,  it  was  felt  that  this  group, 
being  removed  from  the  working  level,  may  be  familiar  only 
with  the  major  types  of  output,  more  common  at  the  working 
level. 

The  group  that  was  chosen  as  being  in  the  best  position 
to  apply  the  weights  were  the  first  level  supervisors.  This 
group  was  large  enough  to  provide  a weighting  scheme  that 
was  reasonably  reliable.  Additionally,  that  group  was  close 
enough  to  the  working  level  to  have  a feel  for  a wide  range 
of  output  types  and  yet  one  level  away  from  the  working  lev- 
el, so  that  it  had  a better  feel  for  output  importance  at 
the  laboratory  level. 

Weighting  Instrument.  The  development  of  the  ques- 
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tionnaire  for  collecting  the  weights  and  determining  the  fi- 
nal scheme  was  then  accomplished. 

Using  the  list  of  output  dimensions  that  was  utilised 

i 

in  the  working  level  self  report,  the  first  level  super- 
visors were  asked  to  place  a weight  on  each  type  of  output 
that  they  felt  best  represented  its  relative  importance  to 
the  laboratory.  In  order  to  provide  a common  point  of  ref- 
erence, these  weights  were  to  sum  to  100  on  each  question- 
naire . 

The  mean  of  the  weights  for  each  type  of  output  obtain- 
from  this  questionnaire  was  then  used  as  a basis  in  the  de- 
termination of  the  final  weighting  scheme  to  be  applied  to 
the  raw  output  reported. 

Predictor  Variable  Measurement 

Cri terla . Chapter  2 pointed  out  the  wide  variety  of 
organisational,  group  and  individual  variables  that  have 
been  investigated  regarding  research  productivity.  As  noted 
in  Chapter  1 however,  the  determination  of  a "best"  set  of 
variables  that  impact  scientist/engineer  productivity  is  be- 
yond the  scope  of  this  research  effort.  Instead  a list  of 
variables  was  chosen  from  those  that  had  been  shown  in  pre- 
vious studies  to  be  correlated  to  a measure  of  scientist/en- 
gineer productivity. 

The  list  of  organisational/individual/group  variables, 
compiled  by  Stahl  (cited  in  Chapter  2)  included  117  differ- 
ent variables  that  had  been  shown  to  be  either  statistical- 
ly significant  or  substantial  in  their  relation  to  research 
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productivity  in  one  or  more  studies.  The  criteria  used  to 
select  from  that  list  a set  of  variables  to  be  investigated 
were  (1)  an  understanding  that  not  all  the  variables  on  that 
list  could  ve  investigated,  (2)  the  desire  to  provide  a bas- 
is of  comparison  to  Stahl's  findings  and  (3)  the  recommenda- 
tions of  the  laboratory  personnel  interviewed.  With  these 
criteria  in  mind,  a preliminary  questionnaire  was  developed 
from  the  one  used  by  Stahl  to  measure  the  variables  included 
in  his  research.  This  questionnaire  was  then  the  basis  for 
discussion  with  the  laboratory  personnel  visited  in  July. 

The  result  of  that  meeting  was  the  identification  of  the  fi- 
nal list  of  predictor  variables  to  be  measured  with  the  ques 
tionnaire . 

Identification.  The  list  of  predictor  variables  that 
was  decided  upon  is  contained  in  Table  IV. 

Table  IV.  Included  Variables 

1 . Age 

2 . Grade/rank 

3 . Education  level 

4.  Length  of  time  in  current  section/group 

5.  Length  of  time  under  current  supervisor 

6.  Years  of  scientific/engineering  experi- 
ence since  first  degree 


7.  Communication  with  other  colleagues  in 


Table  IV  - continued 


11.  Pressure  for  quality  of  output 

12.  Pressure  for  meeting  schedules/deadlines 

13 • Pressure  for  relevancy  of  work  to  the 
Air  Force  mission 

14.  Degree  of  participation  in  setting 
work  goals 

15.  Degree  to  which  quantity  of  output  is 
rewarded 

16.  Degree  to  which  quality  of  output  is 
rewarded 

17*  Degree  to  which  innovative  output  is 
rewarded 

18.  Extent  to  which  supervisor  evaluates 
work 

19 • Extent  to  which  supervisor  controls  work 

20.  Extent  to  which  supervisor  understands 
subordinate's  feelings 

21 . Predominant  nature  of  work 

22.  Frequency  of  attendance  at  professional 
society  meetings 

23*  Organizational  membership  in  the  labor- 
atory, i.e.,  Office  A or  Office  B (de- 
scribed in  the  following  section) 

Measurement . The  instrument  used  to  measure  all  the  pre- 
dictor variables  was  a modification  of  Stahl's  questionnaire. 
This  questionnaire  was  designed  to  obtain  data  for  subject- 
ive variables,  such  as  the  11  perceptual  variables,  and  for 
objective  variables  such  as  age,  grade  and  frequency  of  at- 
tendance at  professional  meetings.  The  questionnaire  is 
contained  in  Appendix  A. 


1 

Subjects 

Sample  Selection.  The  criteria  used  in  the  determina- 
tion of  an  Air  Force  laboratory  to  be  investigated  were  (1 ) 
the  number  of  working  level  scientists/engineers  available 
to  participate  in  the  survey  and  (2)  the  degree  of  coopera- 
tion, i.e.,  response  rate,  that  could  be  anticipated  from 
the  chosen  laboratory . 

The  number  of  Air  Force  R&D  laboratories  located  at 
Wright-Patterson  AFB  initially  seemed  to  indicate  that  the 
selection  of  one  or  more  of  these  laboratories  would  be  i- 
deal  from  the  standpoint  of  close  proximity  and  relative 
ease  of  communication.  However  these  laboratories  had  been 
highly  surveyed  in  the  past,  and  at  the  time  of  this  research, 
several  of  these  laboratories  were  again  being  surveyed  and 
interviewed.  With  the  frequency  of  surveys  being  brought 
to  these  local  laboratories  for  completion,  it  was  felt  that 
an  adequate  response  rate  might  not  be  obtained.  A labora- 
tory less  surveyed  and  researched  might  provide  a more  prompt 
and  adequate  response . 

It  was  brought  to  the  attention  of  the  researcher  that 
an  Air  Force  laboratory,  geographically  located  in  the  west- 
ern half  of  the  United  States,  had  shown  some  interest  in 
the  work  done  by  Stahl.  After  consultation  with  a represent- 
ative of  the  prospective  laboratory  it  was  tentatively  a- 
greed  that  this  laboratory  would  participate  in  the  survey, 
and  during  a meeting  in  June,  the  laboratory  commander  stat- 
ed his  full  cooperation  with  the  research  effort. 

During  the  visit  to  the  laboratory  in  July,  the  research- 
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er  received  a list  of  all  the  personnel  in  the  laboratory. 

This  list  included  approximately  390  working  level  scien- 
tists and  engineers  and  101  first  level  supervisors.  Or- 
ganizationally, the  laboratory  was  divided  into  primary  of- 
fices. For  use  in  this  text,  these  offices  will  be  referred 
to  as  Office  A and  Office  B.  These  offices  had  two  separate 
missions  and  in  recent  years  Office  A had  been  receiving  a 
higher  priority  both  in  funds  and  personnel,  although  Office 
B still  contained  about  twice  as  many  scientists/engineers. 

The  scientists/engineers  in  these  offices  might  be  categor- 
ized as  physical  sciences  oriented  as  opposed  to  a human  re- 
sources or  biomedical  orientation. 

From  the  lists  received,  349  working  level  scientists 
>and  engineers  and  91  first  level  supervisors  were  randomly 
chosen  to  receive  questionnaires.  The  number  of  working 
level  scientists/engineers  selected  was  based  on  an  approxi- 
mate anticipated  response  rate  of  60  percent  to  provide  a 
total  of  200  respondents.  It  was  estimated  that  200  res- 
pondents would  provide  an  adequate  sample  size  for  the  sta- 
tistical analyses  contemplated. 

Sample  Characteristics.  The  overall  response  rate  for 
Part  I of  the  survey,  which  was  the  first  level  supervisor 
questionnaire,  was  65  respondents  out  of  a possible  91 , or 
71.4  percent.  Of  the  65  respondents,  a total  of  12  were  dis- 
carded. One  was  discarded  because  the  weights  did  not  sum 
to  100  as  directed.  The  other  11  had  filled  in  the  "other" 
category  and  applied  a weight  to  that  write-in  dimension. 

There  was  no  consistently  identified  dimension  among  these  11. 
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This  yielded  a usable  response  rate  of  5 8.2  percent  (53 
of  91).  The  range  of  time  as  supervisor  was  from  1 month  to 
12  years,  with  the  mean  time  being  1 year,  11  months.  The 
median  time  was  1 year,  3 months,  with  90.6  percent  of  the 
respondents  having  3.2  years  or  less  as  supervisor  of  their 
current  section  or  group.  These  characteristics  are  dis- 
played in  Table  V. 


Table  V.  Supervisor  Sample 


Response  Rate 
Overall 
Usable 

Time  as  super- 
visor(yrs) 


Range 

Mean 

Median 


71.4*  (65/91) 
58.2*  (53/91) 

.1  to  12. 

1.94 

1.31 


The  overall  response  rate  for  Part  II  of  the  survey, 
the  working  level  scientist/engineer  questionnaire,  was  184 
of  a possible  349  or  52.9  percent.  Of  the  184  respondents, 
49  were  not  included  in  analysis  for  the  following  reasons: 

1 . 26  were  a member  of  their  current  section  or 
group  for  less  than  one  year. 

2.  13  had  failed  to  answer  questions  on  one  or 
more  predictor  variables  on  which  analysis  was  contemplated. 

3 . 8 had  reported  output  for  a particular  dimen- 
sion that  exceeded  five  standard  deviations  from  the  mean  of 
that  dimension,  indicating  a probability  that  they  had  over- 


looked  the  stipulated  two  year  reporting  period.  A check 
of  laboratory  records  to  determine  the  highest  number  of 
each  output  type  produced  over  the  past  two  years  was  not 
made.  Given  that  limitation,  to  insure  that  the  reported 
output  was  relatively  accurate,  these  eight  outliers  were 
discarded. 

4.  2 respondents  did  not  have  academic  degrees 

and  consequently  were  not  considered  scientists/engineers. 

With  these  49  discarded,  the  usable  response  rate  was 
135  for  a rate  of  38.7  percent.  This  sample  size  is  some- 
what less  than  the  anticipated  200.  The  impact  of  this  size 
is  evident  in  the  section  concerning  canonical  correlation 
analysis . 

Age,  education  and  grade  characteristics  of  this  sample 
are  displayed  in  Table  VI. 

Table  VI . Scientists/Engineers  Sample 
Response  Rate 


Overall 

52.7% 

(184/349) 

Usable 

• 

00 

(135/349) 

Age  (years) 

Range 

0 

-t-> 

CM 

57 

Mean 

33-5 

Median 

31.4 

Grade  (.%  of  total) 

GS-7,8,9,  NCO 

05.9 

Lieutenant 

13.3 

GS-10,11 ,12,  Capt 

58.5 

Grade (continued) 


GS-13,  Major 

17.0 

GS-14,  Lt  Col 

05.2 

C i vi lian/Mi li tary 

45/55 

Education  Level 
(<  of  total) 

BS 

31.1 

MS 

48.1 

PhD 

20.7 

Analytic  Methods 

Introduction.  The  Statistical  Package  for  the  Social 
Sciences  (SPSS)  was  the  primary  computer  package  utilized 
for  conducting  the  statistical  analyses  in  this  research. 
SPSS  routines  for  stepwise  multiple  regression  analysis  and 
compilations  of  frequency  distributions  and  histograms,  a- 
long  with  typical  statistical  parameters  were  used  extensive- 
ly. Also  the  nonparametric  Spearman  correlation  routine  was 
used. 

The  Spearman-Brown  Reliability  Prediction  formula  was 
used  to  determine  the  reliability  of  the  weights  received 
from  the  first  level  supervisors  (Ref  25«286). 

Canonical  correlation  analysis  was  used  to  explore  the 
relationship  between  a set  of  9 output  criteria  and  a set  of 
15  individual/organizational  variables  (Ref  23»185). 

Appendix  C includes  the  data  coding  employed  for  use 
with  the  computer  package. 

Validity  of  Output  and  Reliability  of  Weightings.  The 
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self  report  of  output  by  the  working  level  scientists/en- 
gineers did  not  readily  lend  itself  to  a validity  analysis. 
With  a limited  familiarity  of  the  types  of  output  reported,  a 
best  guess  as  to  the  reasonableness  of  the  number  reported 
for  a two  year  time  period  could  be  made.  A better  method 
was  through  comparison  to  the  group  as  a whole.  If  there 
was  an  individual  whose  reported  output  was  five  standard 
deviations  greater  than  the  mean,  then  it  was  assumed  that 
he  had  overlooked  the  two  year  time  period  and  that  ques- 
tionnaire was  discarded. 

The  reliability  of  the  weights  reported  by  the  first  le- 
vel supervisors  were,  on  the  contrary,  readily  subject  to 
the  Spearman- Brown  formula.  The  weighting  scheme  to  be  uti- 
lized was  based  on  the  average  of  the  weights  applied  to  the 
eight  dimensions  of  output.  The  Spearman- Brown  formula  in- 
volves the  determination  of  the  reliability  of  the  mean  of 
k judges,  rating  n items.  In  this  case,  k=53  and  n=8.  As 
the  degree  of  reliability  in  this  formula  is  highly  related 
to  the  number  of  judges  and  because  53  judges  is  relatively 
high,  a reliability  value  of  .7  or  greater  was  viewed  as  ne- 
cessary to  demonstrate  the  reliability  of  the  weights. 

There  are  no  tests  for  significance  of  this  reliability  esti- 
mate, however  levels  of  less  than  .6  are  rarely  reported  in 
literature  using  this  reliability  estimate.. 

After  a determination  of  the  reliability  of  the  weights, 
the  final  weighting  scheme  to  be  employed  was  established. 
This  scheme  was  first  established  by  identifying  the  output 
dimension  which  had  received  the  lowest  average  weight  from 


the  supervisors.  The  weight  applied  to  this  dimension  was 
1 . Each  other  dimension  then  received  a corresponding 
weight  greater  than  or  equal  to  1 based  on  the  comparison  of 
its  overall  average  weight  to  the  average  weight  of  the  smal- 
lest. 

Correlation  Analysis.  Examination  of  the  zero  order 
correlation  coefficients  for  statistical  significance  be- 
tween the  10  criteria  and  the  23  predictor  variables  was 
performed.  This  identified  the  15  predictors  most  consist- 
ently related  to  the  10  criteria. 

Multiple  Regression  Analysis.  It  was  anticipated  that 
several  of  the  predictor  variables  would  be  interrelated, 
thus  partial  correlations  were  determined  through  the  use 
of  multiple  regression  analysis.  As  there  were  10  separate 
criteria  of  productivity  (8  dimensions  and  raw  and  weighted 
sum),  there  were  10  separate  multiple  regression  analyses. 

The  general  form  of  the  linear  model  was 

Yi  = bo  + bjXj  where  i=l,...,l0 

and  j=l , , . . , 23 

Canonical  Correlation  Analysis.  Because  the  10  cri- 
terion variables  were  related,  there  was  a need  to  perform 
multivariate  analysis.  Testing  for  significance  in  canon- 
ical correlation  analysis  requires  pq  degrees  of  freedom, 
with  p equal  to  the  number  of  predictor  variables,  and  q 
equal  to  the  number  of  criterion  variables  (Ref  22ill9). 

With  23  predictor  variables  and  10  criterion  variables, 
135  data  points  do  not  provide  sufficient  degrees  of  free- 
dom to  include  all  possible  variables.  As  pointed  out  later 
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in  Chapter  4,  raw  output  and  weighted  output  are  essential- 
ly the  same  measure  of  productivity.  However  the  fact  that 
the  correlation  coefficient  between  these  two  measures  was 
not  equal  to  1,  perhaps  indicates  that  the  applied  weighting 
scheme  did  slightly  change  the  rank  ordering  of  the  report- 
ed output.  Consequently  weighted  output  was  chosen  in  lieu 
of  raw  output  for  inclusion  in  the  criterion  variables.  By 
eliminating  raw  output  from  the  analysis,  the  criterion  set 
is  reduced  to  9»  thus  increasing  the  number  of  possible  pre- 
dictor variables  from  13  (135/10)  to  15  (135/9). 

The  15  predictor  variables  chosen  were  those  that  show- 
ed significant  zero  order  correlation  coefficients  with  one 
or  more  of  the  output  dimensions. 
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IV.  Presentation  of  Results 
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Introduction 

This  chapter  first  presents  the  means,  standard  devi- 
ations and  distributions  of  the  eight  output  dimensions  as 
reported  by  135  scientists/engineers . Thereafter  follows 
the  reported  weights  and  their  reliability  and  a comparison 
of  the  raw  and  weighted  output  distributions. 

The  sections  on  zero  order  correlations  present  the 

Spearman  correlation  coefficients  among  the  output  dimen- 

1 

sions  and  between  these  dimensions  and  the  23;  predictor 
variables,  identifying  the  15  variables  chosen  to  be  includ- 

i 

ed  in  the  canonical  correlation  analysis. 

Additionally  included  are  the  results  of  the  multiple 

/ 

I 

regression  analyses. 

/ 

/ 

Raw  Output 

Distributions . The  mean  and  standard/  deviation  of  each 
output  dimension,  for  a sample  of  n=135i  is  presented  in 
Table  VII. 

Table  VII.  Output  Means  and  Standard  Deviations 


Dimension  Mean  SD* 
Technical  Report  (TR)**  1.80  2.08 
work  Unit  Planning  Document  (WUFD)**  1.15  1.90 
Journal  Article  (JA)**  0.80  1 .70 
Procurement  Package  (PP)**  2.04  " 3.02 
Evaluation  of  Proposal  (E0P)**  2.30  3. 10 
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Computer  Code  (CC)**  2.16  4.09 


Technical  Presentation  (TP)**  2.97  4.11 
Management  Presentation  (MP)**  2.27  4.12 
* standard  deviation 

**  These  abbreviations  are  used  extensively  throughout  the  re- 
mainder of  the  text  in  reference  to  the  output  dimensions 


A closer  look  at  the  distribution  of  each  output  dimen- 
sion shows  them  all  long  tailed  to  the  right,  following 
a near  exponential  distribution.  These  individual  distri- 


butions are  presented  in  figures  1 through  8. 
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Figure  6.  Technical  Presentations  Distri 


figure  8.  Management  Presentations  Distribution 


Zero  Order  Correlations.  The  highly  skewed  nature  of 
these  distributions  is  evidence  that  an  investigation  of  the 
Pearson  correlation  coefficients  between  the  dimensions 
might  be  misleading  because  of  Pearson's  bivariate  normal 
assumption  (Ref  19*196) . Instead,  Spearman's  rank  order  cor- 
relation coefficients,  which  do  not  assume  bivariate  normal, 
were  investigated.  These  coefficients  are  shown  in  Table 
VIII. 


Table  VIII 

. Correlation  Coefficients  - OutDut 

TR 

WUPD 

JA 

PP 

EOP 

CC  TP  MP 

TR 

1.0 

WUPD 

.197 

1 .0 

JA 

.206 

-.047 

1 .0 

PP 

-.039 

.193 

.014 

1 .0 

EOP 

-.024 

.152 

.197 

• 530 

1 .0 

CC 

.007 

-.110 

.111 

-.288 

-.021 

1.0 

TP 

.300 

.186 

.374 

.094 

QO 

CM 

• 

.167  1.0 

MP 

.174 

.284 

.013 

.437 

.301  ■ 

-.199  .175  1.0 

For 

n=135.  P=. 

05  at  r= 

.I69 

and  p= 

.01  at 

r- . 221 

Although  only  remotely  supported  by  the  above  correla- 
tion coefficients,  there  is  some  benefit,  as  is  shown  later, 
in  grouping  the  eight  dimensions  into  two  broad  based  types. 
One  type,  labelled  technical  output,  includes  technical  re- 
ports (TR),  journal  articles  (JA),  computer  codes  (CC)  and 
technical  presentations  (TP).  The  other  group,  contractual/ 
management  output,  includes  the  remaining  four  dimensions, 
work  unit  planning  documents  (WUPD),  procurement  packages 
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(PP),  evaluations  of  proposals  (EOP)  and  management  presenta- 
tions (MP). 

Table  VIII  does  point  out  the  extent  of  the  relation- 
ships among  the  eight  dimensions,  thus  the  need  for  multi- 
variate analysis.  Prior  to  the  discussion  of  canonical  cor- 
relation analysis,  the  weighting  scheme  and  a comparison  of 
raw  and  weighted  output  is  presented. 


Weighting 


Reliability.  As  previously  noted,  the  reliability  of 
the  weights  applied  to  the  output  dimensions  by  the  first 
level  supervisors  was  determined  by  use  of  the  Spearman- 
Brown  Reliability  Prediction  formula.  The  mean  weights  ap- 
plied to  each  output  dimension  are  listed  in  Table  IX. 

Using  these  weights,  the  Spearman- Brown  formula  resulted  in 
a reliability  of  r'=.88,  for  n=8  and  k=53. 


Table  IX.  Dimensions  and  Mean  Weights 


Dimension 

Mean  Weight 

TR 

20.97 

WUPD 

9.18 

JA 

8.76 

PP 

13.01 

EOP 

9.92 

CC 

15.62 

TP 

12.77 

MP 

9.75 

This  high  a level  of  reliability  was  anticipated  due  to 
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the  high  number  of  judges  used.  A figure  of  less  than  .70 
for  example,  would  lead  one  to  suspect  the  overall  reliabil- 
ity of  the  weights,  but  with  p=.88,  a high  interrater  agree- 
ment is  demonstrated. 

Determination.  With  the  reliability  of  the  above  weights 
determined,  the  final  set  of  weights  were  then  calculated. 

For  ease  of  calculation,  the  weights  in  Table  IX  were  each 
divided  by  8.76,  the  journal  article  weight.  This  resulted 
in  the  final  weighting  scheme  applied  to  the  dimensions  of 
output,  listed  in  Table  X below. 


Table  X.  Dimension  and  Weight  Appli< 
Dimension  Weight  Applied 


WUPD 


These  weights  were  applied  to  the  output  dimensions  re- 
ported by  the  individual  scientists/engineers,  providing  a 
weighted  sum  measure  of  productivity. 

Comparison  of  Raw  and  Weighted  Output 

Distributions.  With  the  application  of  the  weight  to 
each  output  dimension,  a weighted  sum  measure  of  productivi- 
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ty  was  determined  for  each  scientist/engineer.  Por  ease  of 
presentation,  both  the  raw  output  and  weighted  output  mea- 
sures were  grouped  into  increments  of  ten.  These  groups 
were  distributed  as  displayed  in  Figure  9. 


Raw  Output i Mean  = 15*56 

Std  Dev  = 12.10 

Weighted  Outputs  Mean  = 22.76 

Std  Dev  = 16. & 


0= ro--  gPJTT  -3S.1-  4o.l-  j<Tl-  6 0.1-  '70. f- 

10  20  30  ^Butput^Broup  60  70  80 

Figure  9*  Raw  and  Weighted  Output  Distributions 


Zero  Order  Correlations.  The  zero  order  correlation  co- 
efficient between  raw  and  weighted  output  was  somewhat  sur- 
prising. Prior  to  its  determination,  it  was  anticipated 
that  the  weighted  sum  would  provide  a significantly  differ- 
ent measure  of  productivity  than  the  raw  measure . The  under- 
lying assumption  was  that  the  raw  sum  would  not  adequately 
measure  the  scientist/engineer's  productivity  due  to  the  non- 
equality of  content  notion  and  that  a different  measure, 
weighted  sum,  would  take  into  account  this  non-equality  no- 
tion. 

Consequently  it  was  surprising  when  the  zero  order  Spear- 
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man  correlation  coefficient  between  weighted  and  raw  out- 
put was  r=.985.  This  high  a correlation  indicates  that  the 
two  are  essentially  the  same  measure. 

Table  XI  below  shows  the  Spearman  correlation  coeffic- 
ients between  the  eight  individual  dimensions  and  the  two 
summed  measures. 


Table  XI . Correlation  Coefficients  - 
Raw,  weighted  and  Dimensions 


Dimension 

Raw  Output 

Weighted 

Output 

TR 

.3414 

.4298 

WUPD 

.2974 

.2668 

JA 

.3485 

.3324 

PP 

.4965 

.4392 

EOP 

• 5856 

.5174 

CC 

.2270 

.2927 

TP 

.6125 

.6144 

MP 

.5428 

.4793 

Por  n=135.  P= • 05  at  r=.l69  and  p=.01  at  r=.221 
Predictor  Variables 

The  means , standard  deviations  and  zero  order  correla- 
tions for  all  23  predictor  variables  are  presented  in  Appen- 
dix B.  In  the  discussion  that  follows,  the  sample  size  of 
135  is  retained  for  21  of  the  23  predictor  variables.  How- 
ever, due  to  missing  data,  the  sample  size  for  the  organiza- 
tional membership  variable  ( ORGN ) is  124,  and  the  sample 
size  for  the  predominant  nature  of  work  variable  (NATW)  is 
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The  Spearman  zero  order  correlations  between  the  pre- 
dictor variables  and  the  output  criteria  were  investigated 
to  determine  those  variables  which  are  significant  correlates 
of  one  or  more  of  the  output  dimensions.  Additionally,  from 
the  list  of  predictor  variables,  the  15  most  consistent  cor- 
relates would  be  included  in  the  canonical  correlation  anal- 
ysis. The  predictor  list  was  limited  to  15  variables  as  de- 
scribed  in  Chapter  3* 

Table  XII  below  presents  the  variable  abbreviations  and 
meanings  that  are  used  in  the  remainder  of  this  text. 

Table  XII . Variable  Abbreviations 
Individual  Variables 


AGE 

Actual  age  of  scientist/engineer 

GRADE 

Grade/rank  of  scientist/engineer 

EDL 

Educational  level 

Time  Variables 

TIMIS 

Length  of  time  in  current  sec- 
tion or  group 

TIMUS 

Length  of  time  under  current 
supervisor 

YRSE 

Years  of  scientific/engineer- 
ing experience 

Frequency  of  Communi- 
cation Variables 

COMIS 

With  other  scientists/engineers 
in  same  section 

COMOS 

With  other  scientists/engineers 
outside  section 

■ 

COINS 

With  supervisor 

i 
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Table  XII  - continued 


Perception  Variables 
Concerning  Supervisor 

QUANP 

OUALP 

TIMEP 

MISSP 

PART 

OUANR 

QUALR 

INNOR 

EVAL 

CONTR 

EMP 


Pressure  for  quantity  of  output 

Pressure  for  quality  of  output 

Pressure  for  meeting  schedules/ 
deadlines 

Pressure  for  relevancy  of  work 
to  the  Air  Force  mission 

Degree  of  participation  in  set- 
ting work  goals 

Degree  to  which  quantity  of  out- 
put is  rewarded 

Degree  to  which  quality  of  out- 
put is  rewarded 

Degree  to  which  innovative  out- 
put is  rewarded 

Extent  to  which  supervisor  evalu- 
ates work 

Extent  to  which  supervisor  con- 
trols work 

Extent  to  which  supervisor  un- 
derstands feelings 


Other  Variables 

NATH  Predominant  nature  of  work 

FREOA  Frequency  of  attendance  at  pro- 

fessional society  meetings 

ORGN  Organizational  membership  in 

laboratory,  i.e.,  Office  A or 
Office  B 


Correlates  of  Raw  and  Weighted  Output.  (Refer  to  Table 
XIII)  The  most  consistent  positive  correlates  of  both  the 


] 

I 


raw  and  weighted  sum  measures  are  education  level  (EDL), 
frequency  of  communication,  both  within  and  outside  the  sec- 
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tion  (COMIS  and  COMOS),  frequency  of  attendance  at  profes- 
sional meetings  (FREQA),  participation  in  work  goal  setting 
(PART),  and  the  perceived  degree  to  which  quantity  of  out- 
put is  rewarded  (QUANR).  Surprisingly,  age  (AGE)  is  not 
significantly  correlated  to  either  of  the  summed  measures. 

One  variable,  organizational  membership  (ORGN)  was  neg- 
atively correlated  to  both  measures.  This  indicates  one  lab- 
oratory office  more  associated  with  productivity  than  the  o- 
ther.  Grade/rank  (GRADE),  perceived  rewards  for  quality  of 
output  (QUALR)  and  supervisor  empathy  (EMP)  were  positive 
correlates  of  the  raw  sum,  but  although  substantial,  all 
three  correlations  with  the  weighted  sum  were  not  statis- 
tically significant  at  p=.05. 

Only  one  of  the  time  variables,  length  of  time  under  su- 
pervisor (TIMUS),  was  significantly  correlated  to  either  of 
these  measures,  showing  a negative  correlation  with  weighted 
output . 

Correlates  of  Individual  Output  Dimensions.  There  was 
no  a priori  assumption  as  to  the  existence  of  two  or  more 
separate  types  of  output  groups  present  within  the  eight  in- 
dividual dimensions.  An  investigation  of  the  zero  order  cor- 
relates between  the  dimensions  and  the  23  predictor  varia- 
bles however  was  facilitated  by  such  an  assumption.  That  is, 
by  dividing  the  eight  dimensions  into  the  two  broad  based 
types  previously  mentioned,  a degree  of  consistency  was 
shown  concerning  predictor  variables  that  are  correlates  of 
one  type  of  output  group  and  not  the  other. 

Correlates  of  the  Technical  Output  Group.  (Refer  to 
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Table  XIII)  The  technical  output  group  includes  the  dimen- 
sions, technical  report  (TR),  journal  article  (JA),  com- 
puter code  (CC)  and  technical  presentation  (TP).  Education 
level,  although  showing  no  statistically  significant  correla- 
tion to  any  of  the  output  dimensions  in  the  contractual/man- 
agement group,  is  a significant  positive  correlate  to  each  of 
the  output  dimensions  in  the  technical  group.  Similarly 
frequency  of  attendance  at  professional  meetings  (FREQA), 
communication  within  the  section  (COMIS),  participation  on 
setting  work  goals  (PART)  and  time  under  supervisor  (TIMUS) 
have  no  significant  correlation  with  any  of  the  dimensions 
in  the  contractual/management  group,  but  are  significantly 
correlated  to  dimensions  in  the  technical  group.  FREQA  and 
COMIS  are  positive  correlates  of  journal  article,  computer 
code  and  technical  presentation  dimensions.  TIMUS  is  a neg- 
ative correlate  of  technical  reports  and  technical  presenta- 
tions, while  PART  is  a positive  correlate  of  journal  arti- 
cles. 

Correlates  of  the  Contractual/Management  Group.  Six 
variables  were  positive  correlates  of  one  or  more  of  the 
dimensions  in  the  c on trac xual/managemen t group. 

Years  of  scientific/engineering  experience  (YRSE) , de- 
of  supervisor  evaluation  (EVAL) , perceived  rewards  for  qual- 
ity of  output  (QUAIit)  and  perceived  rewards  for  innovative 
output  (INNOR)  were  all  significant  positive  correlates  of 
the  work  unit  planning  document  dimension  (WUPD).  Length  of 
time  in  section  (TIMIS)  was  positive  correlate  of  management 
presentations  (MP),  and  communication  outside  the  section 
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(COMOS),  was  a positve  correlate  of  both  work  unit  planning 
documents  (WUPD)  and  management  presentations  (MP) . Vari- 
ables that  had  significant  correlation  coefficients  with 
procurement  packages  and  evaluations  of  proposals  also  show- 
ed significant  correlations  with  one  or  more  of  the  dimen- 
sions in  the  technical  group.  These  are  identified  next. 

Other  Correlations.  Six  variables  exhibit  somewhat  dif- 
ferent characteristics  than  the  ones  thusfar  noted.  These 
variables  were  shown  to  be  correlates  of  dimensions  from 
both  groups  of  output.  Two  variables,  organizational  mem- 
bership (ORGN)  and  predominant  nature  of  work  (NATW) , under- 
standably correlate  with  both  types  of  output.  Just  as  it 
was  negatively  correlated  with  both  weighted  and  raw  output, 
ORGN  was  negatively  correlated  with  six  individual  output 
dimensions,  statistically  significant  with  journal  articles 
(JA),  procurement  packages  (PP)  and  evaluations  of  proposals 
(EOP).  This  shows  that  one  office  in  the  laboratory  is  more 
associated  with  these  types  of  output  than  the  other.  Pre- 
dominant nature  of  work  (NATW)  was  positively  correlated 
wit  \ procurement  packages  (PP)  and  negatively  correlated 
with  technical  reports  (TR),  computer  codes  (CC)  and  techni- 
cal presentations  (TP).  The  coding  for  NATW  ranged  from  a 1 
for  "research"  to  a 5 for  "contract  monitoring,"  consequent- 
ly these  correlations  are  not  surprising. 

The  other  four  variables  that  cut  across  both  types  of 
output  groups  were  age  (AGE),  grade/rank ( GRADE ) , perceived 
rewards  for  quantity  of  output  (QUANR)  and  supervisor  empa- 
thy (EMP).  GRADE,  QUANR  and  EMP  were  consistent  positive  cor- 


53 


H 


i 


r 


i 

i 


relates  of  one  or  more  of  the  dimensions.  AGE  however  was 
a positive  correlate  of  work  unit  planning  documents  but  a 
negative  correlate  of  computer  codes. 

Uncorrelated  Variables.  Six  variables  were  not  corre- 
lates of  either  raw  and  weighted  output  or  any  of  the  indi- 
vidual dimensions.  None  of  the  perceived  pressure  variables, 
for  quantity,  for  quality,  for  meeting  schedules  or  for  rele- 
vancy of  work  to  the  Air  Force  mission  (QUANP,  QUALP,  TIMEP 
or  MISSP)  were  correlates.  Significantly,  Stahl  found  no 
correlation  between  the  innovation,  time  and  quantity  pres- 
sure variables  and  his  peer  rated  measures  of  innovation  and 
productivity  (Ref  22il52). 

The  degree  of  supervisor  work  control  (CONTR) , which 
is  essentially  a reflection  of  the  scientist/engineer's 
perceived  work  autonomy,  is  not  correlated  with  any  of  the 
productivity  measures.  This  result  is  again  supported  by 
Stahl's  research  (Ref  22il53). 

Frequency  of  communication  with  the  supervisor  (COMWS) 
is  also  uncorrelated  with  any  of  the  productivity  measures. 
This  is  comparable  to  Stahl's  results. 

This  absence  of  significant  correlations  may  lead  one  to 
question  the  value  of  these  six  variables  as  predictors  of 
scientist/engineer  productivity.  Further  comment  on  this 
possibility  is  reserved  for  Chapter  5.  Table  XIII  shows  the 
correlation  coefficients  between  the  23  predictors  and  the 
10  output  criteria. 
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Table  XIII.  Predictor/Criteria  Correlation  Matrix 


Variable 

TR 

JA 

CC. 

EDL 

.2?2** 

.487** 

.213* 

.329** 

.259** 

FREQA 

.165 

.388** 

.217* 

.414** 

.363** 

COMIS 

.008 

.198* 

.064 

.178* 

.205* 

TIMUS 

-.200* 

-.058 

-.144 

-.188* 

-.160 

PART 

.009 

.169* 

.168 

.095 

.248** 

TIMIS 

-.062 

-.073 

-.146 

-.006 

-.121 

YRSE 

.072 

.101 

-.157 

.096 

.079 

EVAL 

-.067 

.032 

.030 

.008 

.114 

QUALR 

-.031 

.093 

.038 

.130 

.191* 

INNOR 

-.016 

.137 

.057 

.118 

.158 

COMOS 

.035 

-.056 

-.057 

.107 

.199* 

AGE 

.013 

-.066 

-.198* 

-.002 

-.005 

GRADE 

.162 

.108 

-.165 

.226** 

.183* 

ORGN 

.035 

-.275** 

-.050 

-.023 

-.206* 

QUA  NR 

.043 

.I87* 

.028 

.137 

.268** 

NATW 

-.254** 

-.150 

-.265** 

-.241** 

.060 

EMP 

.010 

.113 

.187* 

.116 

.182* 

CONTR 

-.121 

-.129 

-.001 

.031 

.041 

COMWS 

-.122 

.104 

.101 

.031 

.110 

QUANP 

.005 

.092 

.062 

-.064 

.074 

QUALP 

.024 

-.016 

.130 

-.101 

-.015 

TIMEP 

-.071 

-.115 

-.025 

-.121 

.024 

MISSP 

-.058 

.008 

.062 

-.063 

-.002 

( 1 ) Raw  output 
* P<-0  5 

**  p <.01 


55 


Table  XIII  - continued 


Variable 

WUPD 

PP 

EOP 

MP 

W0<2) 

EDL 

-.035 

-.017 

.156 

.011 

.259** 

FREQA 

.134 

-.007 

.151 

.140 

.370** 

COMIS 

.126 

.107 

.097 

.086 

.186* 

TIMUS 

.048 

.017 

.051 

-.053 

-.186* 

PART 

.154 

.072 

.082 

.119 

.222** 

TIMIS 

.132 

.020 

.046 

.171* 

-.131 

YRSE 

.262** 

.020 

.072 

.031 

.061 

EVAL 

.209* 

•153 

.060 

.145 

.078 

QUALR 

.211* 

.101 

.098 

.103 

• 157 

INNOR 

.194* 

.100 

.037 

.041 

.137 

COMOS 

.305** 

.127 

.143 

.230** 

.172* 

AGE 

.229** 

-.001 

.073 

.055 

-.027 

GRADE 

.246** 

-.009 

.142 

.147 

.168 

ORGN 

.131 

-.285** 

-.257** 

-.038 

-.181* 

QUANR 

.160 

.186* 

.200* 

.162 

.236** 

NATW 

.013 

.305* 

.149 

.082 

-.086 

EMP 

.101 

.270** 

.177* 

.201* 

.141 

CONTR 

-.005 

.045 

.043 

.023 

.027 

C0MW3 

.022 

.076 

.110 

.131 

.096 

QUANP 

.050 

•135 

.141 

.122 

.051 

QUALP 

.003 

.043 

.005 

.032 

-.031 

TIMEP 

.044 

.095 

.097 

.122 

.002 

MISSP 

.052 

.008 

.056 

.134 

-.031 

(2)  Weighted  output 
* p<.05 

**  p < .01 


Canonical  Correlation  Analysis 

It  was  initially  contemplated  that  the  15  predictor 
variables  most  consistently  correlated  to  the  set  of  out- 
put dimensions  would  be  included  in  the  canonical  corre- 
lation analysis.  However  two  variables,  organizational 
membership  (ORGN)  and  predominant  nature  of  work  (NATW), 
that  would  otherwise  have  been  included,  represented  a 
considerable  source  of  missing  data  on  the  questionnaires. 
In  the  sample  of  135,  11  missing  values  occurred  for  ORGN 
and  18  occurred  for  NATW.  Rather  than  removing  these  data 
points  from  the  sample  and  thus  further  reduce  the  total 


number  of  predictor  variables  that  could  be  included  in 
the  analysis,  it  was  decided  instead  to  exclude  ORGN  and 

NAT''1  from  the  predictor  list.  A closer  look  at  Table  XIII 

< 

shows  a total  of  17  predictor  variables  that  are  signifi- 
cant correlates  of  one  or  more  output  dimensions,  of  which 
two  variables  are  ORGN  and  NATW.  Consequently,  the  remain- 
ing 1 5 variables  made  up  the  list  of  predictors  included 
in  the  canonical  correlation  analysis.  Table  XIV  shows 
the  output  dimensions  and  predictor  variables  included. 

Table  XIV.  Canonical  Correlation  Variables 
Criterion  Variables  Predictor  Variables 


TR 

PP 

TP 

EDL 

TIMUS 

YRSE 

INNOR 

GRADE 

WUPD 

EOP 

MP 

FREQA 

PART 

EVAL 

COMOS 

OUANR 

JA 

CC 

wo 

COMIS 

TIMIS 

QUALR 

AGE 

EMP 

As  there  were  nine  criterion  variables,  there  were  a 
total  of  nine  canonical  roots  formed.  These  results  are 
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shown  in  Table  XV. 


Table  XV.  Canonical  Roots 


ROOTS 

CHI  SQUARE 

d.ft 

£ 

.4310 

69.07 

23 

.000 

.2934 

42.89 

21 

.004 

.2712 

38.75 

19 

.006 

.1951 

26.58 

17 

.067 

.1633 

21.84 

15 

.115 

.1247 

16.31 

13 

.234 

.0968 

12.47 

11 

.331 

.0661 

8.37 

9 

.502 

.0265 

3.30 

7 

.857 

* degrees  of  freedom 

Only  the  first  three  roots  are  significant  at  p=.05, 
consequently  the  remaining  six  roots  are  no  longer  discus- 
sed. 

As  noted  by  Stahl,  an  issue  with  the  use  of  canonical 
loadings  is  the  determination  of  what  magnitude  is  re- 
quired for  a loading  to  be  substantial,  since  there  are  no 
significance  tests  for  each  loading  (Reg  22sl20).  For 
this  research  it  was  decided  that  loadings  of  .2  or  great- 
er were  substantial.  That  level  of  loading  approximates 
the  significance  level  of  the  zero  order  correlations. 
Additionally,  that  level  allows  for  the  verification  of  some 
variables  that  were  significant  in  the  zero  order  correla- 
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tion  analysis.  In  Table  XVI  below,  loadings  of  .2  or  great- 
er are  underlined. 


Table  XVI.  Canonical  Loadings 


Variables 

First 

Second 

Third 

Canonical 

Canonical 

Canonical 

Dimension 

Dimension 

Dimension 

TR 

• 5613 

.0979 

-•0161 

WUPD 

.0137 

•0092 

JA 

.7592 

-.0933 

^081 

PP 

- . 1 541 

. 0242 

♦3034 

EOP 

.1060 

•5^.8 

•1652 

CC 

.4448 

1 094 

-3^,69 

TP 

.5110 

•53H6 

-.2409 

MP 

-.0452 

•336? 

-.0607  ' 

WO 

• 5843 

.4243 

-.0801 

AGE 

-.1712 

.1674 

-.1029 

GRADE 

.1558 

♦76,13 

-.1840 

EDL 

• 7775 

-. 0242 

3028 

TIMIS 

.1705 

.0990 

-.2308 

TIMUS 

-.2410 

•1399 

- . 2642 

YRSE 

-.0202 

1-2623 

,0298 

COMIS 

.2446 

.0604 

-.0666 

COMOS 

-.1413 

.3646 

--3027 

FREDA 

• 5839 

.2141 

-.1493 

PART 

.2302 

-.0620 

-.1870 

QUANR 

.2389 

.0153 

.1080 

59 


* 


dition  to  weighted  output,  the  dimensions  on  the  criterion 
side  interestingly  correspond  to  the  four  dimensions  that 
were  included  in  what  was  labelled  previously  as  the  tech- 
nical output  group.  This  group  of  output  dimensions,  when 
taken  jointly,  might  be  labelled  technical  output  as  the 
loadings  are  relatively  heavy  on  all  four.  ' 

On  the  predictor  side  of  the  first  root,  the  determin- 
ation of  a single  label  that  would  describe  the  various 
kinds  of  variables  is  difficult.  However  a comparison  of 
the  loadings  with  the  zero  order  correlation  coefficients 
previously  described  shows  some  consistent  results. 

Education  level  (EDL)  and  frequency  of  attendance  at 
professional  meetings  ( FREQA ) show  strong  positive  load- 
ings with  the  joint  technical  output  group.  Similarly, 

EDL  was  correlated  with  each  of  these  dimensions  when  taken 
separately  and  FREQA  was  correlated  with  three  of  the  four 
individual  dimensions.  Both  these  variables  seem  to  be  sig 
nificant,  consistent  predictors  of  the  dimensions  in  the 
technical  ourput  group. 
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The  loadings  of  communication  frequency  with  colleagues  in 
the  same  section  (COMIS)  and  length  of  time  under  supervis- 
or (TIMUS)  with  the  joint  criteria  also  support  the  zero  or- 
der correlations.  COMIS  shows  a positive  relationship  with 
the  joint  criteria  just  as  it  was  positively  correlated  with 
journal  articles  (JA)  and  technical  presentations  (TP)  indi- 
vidually. TIMUS,  which  was  found  to  be  negatively  correlat- 
ed to  both  technical  reports  (TR)  and  technical  presenta- 
tions (TP),  shows  a negative  relationship  to  the  joint  cri- 
teria. Similarly,  participation  in  work  goals  and  project 
selection  (PART)  shows  a positive  relationship  to  the  joint 
criteria,  as  it  did  with  journal  articles  (JA)  separately. 

Second  and  Third  Canonical  Dimensions.  An  investigation 
of  the  second  and  third  canonical  dimensions  provided  a less 
clear  indication  of  the  relationships. 

On  the  criterion  side  of  the  sceond  canonical  dimension 
were  weighted  output  (WO),  technical  presentations  (TP),  and 
three  of  the  output  types  included  in  the  contractual/manage- 
ment group.  The  three  contractual/management  types  of  out- 
put were  work  unit  planning  documents  (WUPD) , evaluations  of 
proposals  (EOP)  and  management  presentations  (MP) . 

The  inclusion  of  technical  presentations  in  this  joint 
criteria  does  not  necessarily  invalidate  the  division  of  the 
output  types  into  separate  technical  and  contractmal/manage- 
ment  groups.  Technical  presentations  indeed  may  involve 
both  types  of  orientations.  Generally,  the  loadings  in  the 
second  canonical  dimension  do  provide  support  for  the  con- 
tractual/management and  technical  categories. 
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On  the  predictor  side  of  the  second  canonical  dimension, 
four  variables  have  substantial  loadings,  with  grade  (GRADE) 
being  dominant.  GRADE  was  positively  correlated  with  both 
technical  presentations  and  work  unit  planning  documents  at 
the  zero  order  level,  but  was  not  significantly  correlated 
to  any  of  the  other  dimensions  separately.  Communication 
with  other  scientists/engineers  outside  the  section  (COMOS), 
showing  a positive  relationship  to  this  joint  criteria, was 
found  also  to  be  correlated  to  WUPD  and  management  pre- 
sentations (MP)  separately.  Years  of  scientific/engineering 
experience  (YRSE)  which  shows  a positive  relationship  to  the 
joint  criteria,  was  shewn  to  be  a positive  correlate  of  WUPD 
separately.  Frequency  of  attendance  at  professional  meetings 
(FREPA)  was  not  correlated  with  WUPD,  EOP  or  MP  at  thq  zero 
order  level.  However  it  was  the  highest  correlate  of  all 
predictor  variables  for  both  weighted  output  and  technical 
presentations.  This  may  account  for  its  substantial  load- 
ing in. this  second  canonical  dimension. 

An  investigation  of  the  third  canonical  dimension  shows, 
on  the  criterion  side,  two  output  types  with  positive  load- 
ings, journal  articles  (JA)  and  procurement  packages  (PP). 
Also  on  the  criterion  side  are  two  output  types  with  nega- 
tive loadings,  computer  codes  (CC)  and  technical  presenta- 
tions (TP). 

On  the  predictor  of  this  canonical  dimension  are  educa- 
tion level  (EDL)  and  empathy  (EMP)  showing  a positive  rela- 
tionship. Showing  negative  relationships  are  length  of  time 
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in  section  (TIMIS),  length  of  time  under  current  supervisor 
(TIMUS)  and  frequency  of  communication  outside  section 
(COMOS). 


» 


It  is  unclear  as  to  how  the  criterion  side  of  this  di- 
mension, containing  both  negative 'and  positive  loadings,  re- 
presents scientist/engineer  productivity.  Consequently  the 
relationships  between  the  predictor  variables  in  this  dimen- 
sion and  scientist/engineer  productivity  are  unclear  and  the 
third  canonical  dimension  is  not  further  discussed. 


Multiple  Regression  Analysis 

It  was  anticipated  that  multiple  regression  analysis 
using  each  of  the  output  dimensions  as  a separate  dependent 
variable,  i.e.,  ten  separate  regression  equations,  would 
provide  further  insight  as  to  the  predictor  variables  that 
correlate  with  scientist/engineer  productivity.  It  would 
additionally  provide  further  basis  for  comparison  to  Stahl's 
research.  However  the  analysis  of  residuals  of  the  regre- 
ssions revealed  that  an  assumption  for  significance  testing 
may  well  have  been  violated  due  to  an  abnormality  in  the  plot 
of  the  residuals  (Ref  6). 

Appendix  D shows  a plot  of  the  residuals  for  the  regres- 
sion on  weighted  output.  That  plot  is  typical  of  the  plots 
obtained  for  the  other  nine  regression  models. 

Because  of  this  abnormality  in  the  residuals,  it  was 
decided  that  an  analysis  of  the  partial  correlations  obtain- 
ed from  the  multiple  regressions  would  be  misleading.  There- 
fore they  are  not  presented. 
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Consistent  Correlates 

Tables  XIII  and  XVI  show  respectively  the  zero  order 
correlations  and  the  canonical  loadings  between  the  predictor 
variables  and  the  output  criteria.  For  the  remainder  of  the 
text,  a consistent  correlate  is  defined  as  one  which  is  both 
a significant  zero  order  correlate  and  has  a substantial 
canonical  loading  on  (with)  one  or  more  of  the  output  cri- 
teria . 

Table  XVII  shows  the  predictors  with  significant  zero 
order  correlations  and  substantial  canonical  loadings  on  the 
first  canonical  dimension.  Table  XVIII  shows  the  predictors 
with  significant  zero  order  correlations  and  substantial 
loadings  on  the  second  canonical  dimension. 


Table  XVII.  Consistent  Correlates  - 
First  Canonical  Dimension 


Variable 

Zero  Order  Correlation' Coefficient 

■ft 

Loading 

JR  JA 

CC  IP 

WO 

EDL 

00 

-d- 

• 

CM 

CM 

• 

.213  .329 

.259 

.778 

FREQA 

00 

00 

• 

.217  .414 

• 370 

.584 

COMIS 

.198 

.178 

.186 

.245 

TIMUS 

-.200 

-.188 

.186 

-.241 

PART 

.169 

.222 

.230 

QUA  NR 

.187 

.236 

.239 

* P=.0 5 at  r=.l69  and  p=,01 

at  r=.221 

l 
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Variable 

Table 

XVIII.  Consistent  Correlates  - 
Second  Canonical  Dimension 

Loading 

Zero 

Order  Correlation  Coefficient-r* 

WUPD 

EOP  TP  — MP  WO 

GRADE 

.246 

.226 

.761 

YRSE 

.262 

.269 

COMOS 

• 305 

CM 

rH 

• 

o 

CM 

• 

.365 

FREQA 

.370 

.214 

*p=.05  at 

r=.l69  and  p=.01  at  r=.22l 

The  predictor  variables  contained  in  Tables  XVII  and 
XVIII  are,  in  a relative  sense,  stronger  predictors  of  sci- 
entist/engineer performance  than  the  other  organizational 
and  individual  variables  investigated.  This  is  evidenced 
by  the  consistency  of  their  significant  correlations  with 
both  the  summed  and  individual  measures  of  productivity, 
separately  and  in  the  joint  canonical  dimensions. 

The  implication  of  this  and  the  other  findings  pre- 
sented in  this  chapter  are  further  discussed  in  Chapter  5. 
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Chapter  V.  Discussion  and  Conclusions 


This  chapter  provides  comments  and  suggests  implications 
of  the  results  that  were  presented  in  Chapter  4. 

Weighting  Scheme 

As  noted  in  Chapter  3*  there  was  a question  in  the  mind 
of  the  researcher  as  to  which  sample  of  laboratory  personnel 
could  provide  a reliable  set  of  weights  to  be  applied  to  the 
eight  different  output  dimensions.  As  shown,  the  use  of  the 
first  level  supervisors  for  this  purpose  did  provide  a set 
of  weights  that  demonstrated  high  interrater  agreement.  How- 
ever the  implication  should  not  be  made  that  the  weights  re- 
ceived from  this  sample  is  the  only  weighting  scheme  that 
might  have  been  determined.  Nor  should  it  be  implied  -that 
the  weighting  scheme  is  the  one  that  most  accurately  reflects 
the  "true  value"  of  each  output  type  to  the  laboratory  as  a 
whole.  It  might  be  postulated  that  the  working  level  scien- 
tists/engineers themselves,  or  the  higher  levels  of  supervis- 
ion could  similarly  demonstrate  high  interrater  agreement, 
would  one  then  choose  the  weighting  scheme  that  shows  the 
highest  reliability?  Certainly  there  would  still  be  doubt 
as  to  the  validity  of  the  weights.  The  subjective  nature  of 
any  weighting  scheme  contains  this  degree  of  doubt,  regardless 
of  how  rigorously  it  is  defended.  It  is  important  to  note 
that  the  researcher  has  carefully  avoided  stating  that  the 
best  set  of  weights  has  been  employed, but  only  that  a statis- 
tically reliable  set  has  been  used. 


Implications  of  Correlations 

Raw  and  Weighted  Output.  It  was  interesting  to  find 
these  two  measures  to  be  so  highly  correlated  (r=.985).  As 
can  be  gathered  from  Chapter  2,  numerous  researchers  were  so 
concerned  with  the  non-equality  of  content  notion,  that  the 
use  of  a peer  or  supervisor  rating  in  conjunction  with  a 
tangible  output  count  was  quite  common  in  determining  an  over 
all  productivity  measure. 

The  weighting  scheme  provided  by  the  first  level  super- 
visors indicate  that  this  non-equality  of  content  notion 
(from  the  standpoint  of  importance  to  the  laboratory)  is  re- 
cognised, at  least  by  the  first  level  supervisors,  what  the 
high  correlation  between  raw  and  weighted  output  seems  to  im- 
ply is  that  those  individuals  producing  low  (relatively)  im- 
portance output,  do  so  in  such  quantities  that  their  overall 
productivity  ranking  changes  very  little  when  the  weighting 
scheme  is  applied.  Conversely,  those  individuals  producing 
high  importance  output,  also  produce  other  output  in  such 
quantities  so  that  their  ranking  in  raw  output  is  relatively 
the  same. 

Predictor  Variables  and  Output  Dimensions.  It  is  import 
ant  to  note  initially  that  no  single  variable  was  shown  to  be 
associated  significantly  with  all  eight  of  the  individual 
output  dimensions.  The  variables  identified  in  Chapter  4 as 
being  consistent  correlates  of  one  or  more  of  the  output  di- 
mensions can  be  considered  those  that  are  most  highly  asso- 
ciated with  the  quantitative  measures  of  productivity  invest! 


gated  in  the  research.  Educational  level  was  shown  to  be  a 
consistent  correlate  of  weighted  output  and  four  of  the  indi- 
vidual dimensions  both  separately  and  with  the  joint  criteria 
of  the  first  canonical  dimension.  The  four  individual  dimen- 
sions and  the  first  canonical,  with  which  educational  level 
was  associated,  were  technical  types  of  output.  This  seems 
to  imply  that  individuals  with  higher  academic  degrees  may 
prove  to  be  most  productive  in  laboratory  assignments  requir- 
ing technical  types  of  output. 

The  frequency  of  communication  variables,  with  colleagues 
in  the  section  (COMIS)  and  with  scientists/engineers  outside 

j 

the  section  (COMOS)  were  both  shown  to  be  positive  correlates 
of  weighted  output.  It  is  interesting  to  note  however  that 
these  two  variables  were  shown  to  be  correlates  of  different 
individual  output  dimensions.  COMIS  was  positively  associ- 
ated with  journal  articles  and  technical  presentations  while 
COMOS  was  associated  with  work  unit  planning  documents  and 
management  presentations.  Perhaps  the  communication  within 
a section  reflects  a sharing  of  homogenous  expertise  that 
might  be  required  for  a journal  article  or  technical  present- 
ation, while  the  external  communication  reflects  a degree  of 
inter-section  coordination  and  information  gathering  required 
for  planning  of  projects  and  management  presentations. 

The  association  of  the  frequency  of  attendance  at  profes-  i 

sional  meetings  variable  (FREOA)  with  performance  might  indi- 
cate to  the  laboratory  manager  the  importance  of  keeping  his 
scientists/engineers  current,  especially  in  the  technical 
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Two  additional  variables  which  were  consistent  corre- 
lates of  weighted  output  and  which  should  be  of  interest  to 
an  RAD  laboratory  manager  are  participation  on  work  goals 
and  length  of  time  under  supervisor,  PART  and  TIMUS  respect- 
ively. The  positive  association  of  PART  with  performance  re- 
flects the  greater  productivity  of  a scientist/engineer  who 
has  an  active  voice  in  his  project  selection  and  work  goals. 
The  negative  association  of  TIMUS  with  performance  may  indi- 
cate that  productiveness  is  enhanced  by  a relatively  frequent 
change  in  supervisor. 

Rewards  for  quantity  of  output  (QUANR)  was  found  to  be 
a consistent  correlate  of  weighted  output.  Similar  to  Stahl, 
one  might  speculate  as  to  the  nature  of  the  rewards  avail- 
able to  scientists/engineers  in  a Civil  Service  or  Air  Force 
employment.  If  in  fact,  a perceived  reward  for  quantity  of 
output  is  the  opportunity  to  attend  professional  society 
meetings,  then  the  laboratory  manager  might  well  be  aware 
of  the  associations  reported  in  this  research. 

Two  other  variables  that  were  found  to  be  consistent  cor- 
relates of  one  or  more  individual  output  dimensions  were 
grade/rank  (GRADE)  and  years  of  scientific/engineering  ex- 
perience (YRSE).  GRADE  was,  in  addition  to  being  a consist- 
ent positive  correlate  of  work  unit  planning  documents  and 
technical  presentations,  a significant  positive  correlate  of 
raw  output.  Similarly,  Stahl  found  grade/rank  to  be  a posi- 
tive correlate  of  productivity  and  innovation,  although  not 
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as  consistent  a one  as  some  others  previously  mentioned. 

YRSS,  although  a consistent  correlate  of  work  unit  planning 
documents,  did  not  show  any  significant  correlations  with  raw 
or  weighted  output,  and  in  a relative  sense  is  less  a predict- 
or of  performance  than  those  mentioned. 

Perhaps  equally  important  is  an  understanding  of  the 
variables  that  show  little  or  no  association  with  perform- 
ance. Interestingly,  the  degree  to  which  the  supervisor  con- 
trols the  work  (CONTR)  was  a correlate  of  none  of  the  prod- 
uctivity criteria.  This  variable,  which  is  essentially  the 
scientist/engineer ' s perception  of  his  work  autonomy,  was 
similarly  reported  by  Stahl. 

None  of  the  pressure  variables,  for  quantity/quality  ~f 
output,  for  meeting  schedules,  or  for  relevancy  of  work  to 
the  Air  Force  mission,  showed  any  significant  correlations. 

The  implication  of  the  value  of  pressure  as  a method  to  in- 
crease performance  should  be  obvious. 

Age  (AGE)  was  another  variable  for  which  no  significant 
consistent  correlations  were  found.  At  the  zero  order  level, 
AGE  was  correlated  negatively  with  computer  codes  and  posi- 
tively with  work  unit  planning  documents.  Hov  ver  its  value 
as  a predictor  of  overall  performance  was  not  substantiated 
by  this  research. 

Shortcomings  of  This  Research 

Although  not  totally  supported  by  the  canonical  correla- 
tion analysis,  some  of  the  associations  of  predictor  varia- 
bles and  output  dimensions  seem  to  indicate  a pattern.  The 
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application  of  additional  statistical  techniques  to  this  data 
might  indeed  provide  greater  clarity  concerning  the  existence 
of  categories  of  output  within  the  eight  output  dimensions  in- 
vestigated. The  problems  encountered  with  the  residuals  of 
the  multiple  regression  analysis  prevented  that  technique 
from  shedding  any  additional  light  on  the  nature  of  the  rela- 
tionships . 

The  criterion  used  for  determining  a weighting  scheme 
was  based  on  the  raters'  perception  of  the  importance  of  the 
output  to  the  laboratory.  There  was  no  investigation  to  de- 
termine that  a better  criterion  could  have  been  used. 

Recommendations  for  Future  Research 

As  an  outgrowth  of  this  research,  there  appears  to  be 
at  least  two  topics  in  the  area  of  measuring  Air  Force  R&D 
scientist/engineer  productivity  that  is  worthy  of  further 
research. 

If  the  trend  for  a tighter  military  budget  continues, 
and  an  equal  or  greater  amount  of  effort  will  be  required 
from  relatively  fewer  resources,  Air  Force  scientists  and  en- 
gineers will  be  required  to  be  both  managers  and  scientists. 
The  types  of  output  that  a scientist/engineer  will  perhaps 
be  required  to  produce  will  be  budgeting,  planning  and  more 
management  related  output,  as  well  as  the  more  traditional 
technical  and  research  types.  As  this  develops,  the  need 
for  understanding  the  types  of  individual  and  organizational 
variables  that  associate  with  different  types  of  output  will 
increase.  Consequently,  the  identification  of  these  dimen- 
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sions  of  output  and  their  associated  variables  will  be  impor- 
tant in  trying  to  enhance  productivity. 

The  trend  of  broad  types  of  output  is  partially  sub- 
stantiated by  this  research.  This  research  needs  to  be  re- 
plicated in  other  Air  Force  R&D  laboratories  with  addition- 
al statistical  techniques  utilized  to  further  identify  the 
broad  output  categories  and  their  associated  variables. 

A second  area  is  an  outgrowth  of  the  weighting  deter- 
mination accomplished  in  this  research,  and  involves  a deter- 
mination of  a multiple  set  of  weights  for  a given  laboratory. 

These  weights,  gathered  from  samples,  organizationally  lo- 
cated at  different  levels,  might  provide  an  indication  of 
how  well  the  various  levels  concur  in  their  view  of  what  type 
of  output  is  important  to  the  laboratory.  The  different 
weights  would  be  a form  of  organizational  feedback,  testing 
how  well  output  emphasis  at  the  top  of  an  organization  com- 
pares to  that  down  through  to  the  working  level. 
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APPENDIX  A 


PART  It  SUPERVISOR  QUESTIONNAIRE 
PART  II t SCIENTIST/ENGINEER  QUESTIONNAIRE 


DEPARTMENT  OF  THE  AIR  FORCE 

AIK  FORCE  INSTITUTE  OF  TECHNOLOGY  < AU  > 
WRIGHT  PATTFRSON  AIR  rORCC  BASE.  OHIO  45433 


A?.lioF®  ENS/ILt  'Coser/525^9 
•object:  Survey  Information 


AUfi  11  m 


/a 


TO:  Respondent 


1 .  The  attached  questionnaire  is  Part  I of  a two-part 
survey,  designed  to  provide  information  necessary  for  my 
masters  degree  thesis  effort.  Part  I is  t(^b^comp^ted 
by  section  chiefs  and  group  leaders  in  the 


2.  approved  the  administration  of  this 
questionnaire  in^SHand  made  mention  of  it  in  his  staff 
meeting  and  at  the  recent  Commander's  Call. 

3.  If  you  are  a section  chief  or  group  leader,  please 
complete  the  attached  questionnaire,  place  it  in  the  re- 
turn envelope  provided  and  mail  as  soon  as  possible.  Let 
me  emphasize  that  anonymity  of  all  respondents  will  be 
maintained.  The  number  in  the  lower  left  comer  of  the 
privacy  statement  is  the  approved  survey  control  number 
and  in  no  way  identifies  the  questionnaire  respondent. 

4.  Thank  you  for  your  cooperation  and  support  in  com- 
pleting this  thesis  effort. 


C.AZlt* 


Michael  C.  Koser,  lLt,  USAF 
Graduate  Student 

Department  of  Systems  Management 
School  of  Engineering 
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(circle  answer) 


PART  I:  SUPERVISOR  QUESTIONNAIRE 

A.  You  are  organizationally  under t HU 

B.  Length  of  time  as  supervisor  of  present  section/groupi  yrs. n 

C . Listed  below  are  types  of  output  of  the  scientist/engineer  in  an 
R&D  laboratory.  If  it  is  assumed  that  not  all  of  these  types  of 
output  are  equal  in  importance  to  the 

then  each  would  carry  a different  we  ignta^s^neasureo^outpu^^^ 
For  example,  two  work  unit  planning  documents  may  be  equal  to  one 
proposal  evaluation  in  importance . The  proposal  evaluation  would 
be  weighted  twice  that  of  a work  unit  planning  document. 


In  the  blank  to  the  right  of  each  of  the  following  types  of  out- 
put, please  fill  in  a weight  which  you  feel  best  represents  the  re- 
lativ^importance  of  that  type  of  output  to  the 
HHHIi  fthe  sum  of  these  weights  should  eoual  100. 

1.  Technical  Report  (including  test  reports)  

2.  Work  Unit  Planning  Document  

3.  Journal  Article  (either  submitted  or  published)  

4.  Procurement  Package  

5.  Evaluation  of  Proposal  

6.  Computer  Code  

7.  Technical  Presentation  including  those  made  out- 
side the  laboratory)  

8.  Management  Presentation  (budget  review,  for  example)  

9.  Other  (please  specify)  


Sum  = 


DEPARTMENT  OF  THE  AIR  FORCE 

AIR  FORCE  INSTITUTE  OF  TECHNOLOGY  IAU) 
WRIGHT  PATTERSON  AIR  FORCE  BASE.  OHIO  45433 

ENS/ILt  Koser/525^9  A 

subject:  Survey  Information 

TO:  Respondent 


1 . The  attached  questionnaire  is  Part  II  of  a two-part 
survey,  designed  to  provide  information  necessary  for  my 
masters  degree  thesis  effort.  Part  II  is  to  be  completed 
by  HHI  scientists/engineers  in  non-supervisory  positions. 

2.  ■■■■■■(Jias  approved  the  administration  of  this 
questionnaire  in^ISBand  made  mention  of  it  in  his  staff 
meeting  and  at  the  recent  Commander's  Call. 

3.  If  you  are  not  a section  chief  or  group  leader,  please 
complete  the  attached  questionnaire,  place  in  the  return 
envelope  provided  and  mail  as  soon  as  possible.  Let  me 
emphasize  that  anonymity  of  all  respondents  will  be  main- 
tained. The  number  in  the  lower  left  corner  of  the  pri- 
vacy statement  is  the  approved  survey  control  number  and 
in  no  way  identifies  the  questionnaire  respondent. 

4.  Thank  you  for  your  cooperation  and  support  in  com- 
pleting this  thesis  effort. 

Michael  C.  Koser,  lLt,  USAF 
Graduate  Student 
Department  of  System  Management 
School  of  Engineering 


Strtmgtb  Through  Kuowltdgt 


PART  II.  SCISNTISTANGINEER  QUESTIONNAIRE 


: 


,1 


A . Output  Measures 

During  the  past  two(?)  years,  how  many  of  each  of  the  following 
have  you  authored,  co-authored,  prepared,  presented,  etc?  (fill 
in  the  blank) 

1.  Technical  Reports  (including  test  reports)  

2.  ’’’ork  Unit  Planning  Documents  

3*  Journal  Articles  (either  submitted  or  published)  

4.  Procurement  Packages  

5*  Evaluations  of  Proposals  

6.  Computer  Codes  

7.  Technical  Presentations  (including  those  made 

outside  the  laboratory)  

8.  Management  Presentations  (budget  review,  for  example) 

9*  Other  (please  specify)  
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B.  Individual/Organizational  Variables 
aT  Please  complete  the  following* 

Age  Current  Grade/Rank  

Education  Level  (circle  highest  degree)  BS  MS  PhD  Other (specify) 


Time  in  current  sect  ion/group  i yrs,  mos. 

Length  of  time  under  current  supervisor i yrs,  mos. 

Years  of  scientific/engineering  experience (research,  consulting, 
etc.)  since  first  degree « 

Nature  of  Current  work(#  of  time  per  category) « 

Research  

Exploratory  Development  

Engineering  Development  

System  Support  

Contract  Monitoring  

Your  freauency  of  communication  (number  per  week)  withi 

Scientists/engineers  within  your  section/group  on  technical 
matters.  

Scientists/engineers  outside  your  section/group  on  technical 
matters . 

Supervisor  on  technical  matters.  

Your  frequency  of  attendance  at  national  or  international  profes- 
sional society  meetings  (number  per  year).  


You  are  organizationally  under: 


(circle  answer) 


B.  Please  complete  the  following  by  placing  an  X above  what 
you  feel  is  the  appropriate  category.  If  you  have  been  working  for 
your  current  supervisor  for  less  than  six(6)  months,  answer  these 
questions  with  respect  to  your  previous  supervisor. 

1 . Amount  of  pressure  exerted  upon  yor  by  your  supervisor  fon 
a.  Quantity  of  output 


1 2 
No  pressure- 
quantity  of 
output  never 
mentioned 


3 T"  T 


~r  — 

Extensive  pressure - 
quantity  of  output 
always  stressed 


b.  Quality  of  output 


No  pressure - 
quality  of 
output  never 
mentioned 


TT  ~ 

Extensive  pressure- 
duality  of  output 
always  stressed 


c.  Meeting  deadlines  and  schedules 


1 2 “ 
No  pressure- 
deadline  s/sched- 
ules never  men- 
tioned 


Extensive  pressure - 
deadlines/schedules 
always  stressed 


d.  Relevancy  of  work  to  the  Air  Force  mission 


1 2 
No  pressure - 
relevancy  of 
work  to  AF  mis- 
sion never  men- 
tioned 


T ~ 

Extensive  pressure- 
relevancy  of  work 
to  AF  mission  al- 
ways stressed 


2.  Extent  of  participation  with  supervisor  on  goal  setting 
(including  project  selection)  concerning  your  work 


~r  ~ — 

No  participation- 
you  set  uour  own 
goals  or  he  sets 
them  for  you 


T ~ 

Total  participation- 
he  always  actively 
shares  this  function 
with  you 
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V i 


I i 


3.  Extent  to  which  supervisor  rewards « 

a.  Quantity  of  output 


T“ 


Never  rewards 
quantity  of 
output 


T ~9~ 

Always  provides 
meaningful  rewards 
for  quantity  of 
output 


b.  duality  of  output 


T"  — 

Never  rewards 
quality  of 
output 


~V 


nr 


T — 

Always  provides 
meaningful  rewards 
for  quality  of 
output 


c.  Innovative  (new  and  useful)  output 


2 3 

Never  rewards 
innovative  out- 
put 


~TT 


~~r 


nr  t 

Always  provides 
meaningful  rewards 
for  innovative  out- 
put 


4.  How  thoroughly  does  your  supervisor  evaluate  your  work? 


T ~T  ~T 

No  evaluation- 
he  never  evaluates 
your  work 


T 


T T~ 

Total  evaluation- 
he  always  evaluates 
your  work  in  detail 


5.  Extent  of  control  exerted  by  supervisor  over  your  work 


Absolute  control- 
you  have  no  free- 
dom concerning 
your  work 


nr 


t 


T — 

No  control- 
you  have  total  free- 
dom concerning  your 
work 


6.  Extent  of  supervisor's  empathy  (i.e.,  he  understands  your 
feelings) 


T T ' 

No  empathy- 
he  never  under- 
stands your 
feelings 


TT 


~r 


nr  T 

Complete  empathy - 
he  always  undertands 
your  feelings 
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APPENDIX  B 


MEANS,  STANDARD  DEVIATIONS  AND 


CORRELATION  MATRIX  FOR 


PREDICTOR  VARIABLES 


STANDARD 


VARIABLE 

MEAN 

DEVIATK 

AGE 

33.4? 

7-26 

GRADE 

3.  02 

•87 

EDL 

1.90 

•?2 

TIMIS 

3-20 

3*02 

TIMUS 

1.39 

1.16 

YRSE 

8.23 

6.11 

NATW 

3-17 

I.47 

COMIS 

3.51 

2.31 

COMOS 

1.99 

1.42 

COMWS 

1. 81 

1.31 

FREOA 

1.46 

00 

• 

ORGN 

1-59 

.49 

QUANP 

3-84 

2.25 

QUALP 

4.  87 

2.50 

TIMEP 

5.51 

2.31 

MISSP 

5.67 

2.30 

PART 

5.27 

2.80 

QUA  NR 

4.70 

2.48 

QUALR 

5.45 

2.33 

INNOR 

5-59 

2.36 

EVAL 

5.55 

2.21 

CONTR 

6.03 

2.16 

EMP 

6.47 

2.08 

84 


AGE 

GRADE 

EDL 

TIMIS 

TIMUS 

YRSE 

AGE 

1 .0 

GRADE 

.633** 

1 .0 

EDL 

-.128 

.086 

1 .0 

TIMIS 

.323** 

.285** 

-.168 

1.0 

TIMUS 

.206* 

.094 

-.179* 

.364** 

1 .0 

YRSE 

.734** 

.672** 

-.028 

.396** 

.257** 

■ 1 . 0 

NATW 

.161 

-.002 

-.180* 

.026 

.161 

.054 

COMIS 

.072 

.106 

.113 

-.018 

-.013 

.208* 

COMOS 

.226** 

.187* 

-.128 

.147 

.118 

.360** 

COMWS 

-.008 

-.088 

.060 

-.022 

.066 

-.034 

FREDA 

-.028 

.036 

.289** 

.010 

-.106 

.053 

ORGN 

.112 

.108 

-.241** 

.083 

.130 

.128 

QUANP 

-.120 

-.095 

-.013 

-.028 

-.002 

-.133 

DUALP 

-.143 

-.176* 

-.013 

-.043 

.032 

-.223** 

TIMEP 

-.139 

-.134 

-.061 

-.007 

-.012 

-.161 

MISSP 

- .162 

-.114 

.118 

-.137 

-.095 

-.223** 

PART 

-.115 

-.142 

.048 

-.176* 

-.024 

-.136 

QUANR 

-.121 

-.047 

.088 

-.095 

-.051 

-.075 

QUALR 

-.098 

-.042 

.055 

.021 

.077 

- .066 

INNOR 

-.120 

-.072 

.048 

.015 

.040 

-.085 

EVAL 

-.157 

-.096 

-.015 

-.127 

-.018 

-.164 

CONTR 

.026 

-.088 

-.117 

-.078 

-.022 

.052 

EMP 

-.055 

.002 

.052 

-.109 

r'l 

0 

• 

1 

.059 

* P<  .05 

**  p<.01 
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UNCLASSIFIED  GSM/SM/76D-33  NL 


2*  2 

%35752 


DATE 

FILMED 

3-77 


[1 

NATW 

COMIS 

COMOS 

COMWS 

FREQA 

ORGN 

1 

1 | NATW 

1 .0 

COMIS 

n 1 

169 

1.0 

COMOS 

027 

. 481** 

1 .0 

COMWS 

• 04? 

. 517** 

. 299** 

1 .0 

FREQA 

278** 

.120 

-.037 

.069 

1 .0 

ORGN 

. Oil 

017 

•135 

-.042 

-.203* 

1 .0 

1 QUANP 

. 088 

. 032 

. 019 

.108 

.103 

-.225* 

QUALP 

. 064 

. 083 

.046 

.203* 

.054 

-.114 

TIMEP 

. 112 

• 074 

. 030 

.160 

-.019 

-.121 

MISSP 

• 063 

. 046 

-.  005 

.129 

•053 

-.202* 

I . PART 

- 013 

. 136 

. 043 

.303** 

.092 

-.094 

[1  QUANR 

015 

. 074 

.186* 

.108 

.156 

- . 205* 

I QUALR 

• 052 

. 096 

. 233** 

.132 

.140 

-.073 

1 INNOR 

• 027 

•159 

.195* 

.171* 

.162 

-.128 

| EVAL 

. 060 

. 094 

. 110 

. 216* 

.087 

•057 

: i 

\ CONTR 

-13 

. 043 

.130 

-.043 

.038 

.165 

EMP 

.042 

.034 

.187* 

.012 

-.028 

.072 

* P < 

• **  P< 

[ 

>Orl 
O O 
• • 

H 

i.-. 

. 

j 

■ 

! I 

? t : 
III 

j 

1 

QUANP 

QUALP 

TIMEP 

MISSP 

PART 

QUANR 

QUANP 

1 .0 

QUALP 

.642** 

1.0 

TIMEP 

.646** 

. 596** 

1.0 

MISSP 

•391** 

.482** 

.590** 

1 .0 

PART 

.081 

.348** 

.178* 

.303** 

1 .0 

QUANR 

•322** 

.207* 

.116 

.205* 

.411** 

1.0 

QUALR 

• 079 

.307** 

.114 

.298** 

. 506** 

•753** 

INNOR 

.117 

.348** 

.174* 

. 286** 

. 505** 

.729** 

EVAL 

• 159 

.362** 

.265** 

. 340** 

.470** 

. 522** 

CONTR 

-.371** 

-.197* 

-.326** 

-.206* 

• 135 

.036 

EMP 

-.257** 

-.122 

-.132 

-.008 

.316** 

.296** 

* P < 
**  P < 

.05 

.01 

QUALR 

INNOR 

EVAL 

CONTR 

EMP 

QUALR 

1 .0 

INNOR 

.915** 

1 .0 

EVAL 

.706** 

.723** 

1 .0 

CONTR 

.163 

.122 

-.018 

1 .0 

EMP 

.407** 

.352** 

.286** 

.367** 

1 .0 

I 

1 

'• 


i 


VARIABLE 

VALUE 

CODE 

AGE 

Age 

Actual  age 

J 

GRADE 

Grade/Rank 

(1) GS-7.8,9.NC0 

( 2 ) Lieu tenant 

(3) GS-10,11 ,12 
Captain 

(4) GS-13.Major 

(5) GS-14,  LtCol 

EDL 

Highest  degree 

(1) B.S. 

(2) M.S . 

(3) PhD 

* m 

•i 

TIMIS 

Number  of  years 
in  section 

Actual  years 

• 

TIMUS 

Number  of  years 
under  supervisor 

Actual  years 

YRSE 

Number  of  years 
of  experience 

Actual  years 

NATV» 

Predominant  na- 
ture of  work 

(1 )Research 

( 2 Exploratory 
development 

(3) Engineering 
development 

(4) Sy8tem  sup- 
port 

(5 Contract  mon- 
itoring 

j 

1 

COMIS 

Number  of  times 
per  week 

(1) 0-3 

(2) 4-7 

(3) 8-11 

COMOS 

«• 

(4) 12-15 

(5) 16-19 

(6) 20-23 

COMA’S 

•• 

(7) 24-27 

(8) 28-31 

(9) 32  or  more 

« 

PREOA 

Number  per  year 

(DO 

(2)1-2 

(3) 3-4 

(4) 5-6 

(5) More  than  6 

« 

ORGN 

Organizational 

membership 

(1) Office  A 

(2) 0ffice  B 

3 
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Vita 


Michael  Curtis  Koser  was  bom  on  28  April  1951  in 
Lancaster,  Pennsylvania.  He  graduated  from  high  school  in 
Elizabethtown,  Pennsylvania  in  1969  and  attended  the  United 
States  Air  Porce  Academy  from  which  he  received  the  degree 
of  Bachelor  of  Science  and  a commission  in  the  USAF  in 
June,  1973.  His  first  assignment  was  in  the  GBU-15  System 
Program  Office,  Eglin  APB,  Florida,  where  he  served  as  Data 
Management  Officer  until  entering  the  School  of  Engineering, 
Air  Porce  Institute  of  Technology,  in  September  1975. 

Permanent  address*  Box  27 

Bainbridge,  Pa  17502 


~'"T'  * - ^ 


UNCLASSIFIED 

SECURITY  CL  MU  FI  CATION  or  TW)»  N API  |WR—  BW  SnlgrqQ 

REPORT  DOCUMENTATION  PAGE 

1 n^nont  NUMSIN  I*.  OOVT  ACCESSION  m 

GSM/SM/76D-33 

4.  TITLE  <m4  IvMlltJ 

QUANTITATIVE  SCIENTIST/ENGINEER 
PRODUCTIVITY  AND  SOME  ASSOCIATED  INDI 
VIDUAL  AND  ORGANIZATIONAL  VARIABLES 

?.  AUTHOR!*! 

Michael  C.  Koser.  lLt,  USAF 

I.  MnroMMNa  oroanization  name  ano  adonis* 

Air  Force  Institute  of  Technology (AFIT/BN 
Wright-Patterson  AFB,  Ohio  45433 

II.  CONTNOLLINO  OFFICE  NAME  AND  ADONESS  " 

Air  Force  Institute  of  Technology (AFIT/En 
Wright-Patterson  AFB,  Ohio  45433 


IS.  OISTNIOUTlON  ST  ATCMENT  (it  IM,  ftoparl) 

Approved  for  public  release;  distribution  unlimited 


[IT.  DISTRIBUTION  STATEMENT  (at  Ola  ok*  tract  anlarad  In  Slack  20.  II  OWaranl  tram  Kaport) 


read  wsntucnoNt  1 

BEFORE  COMPLETINO  F04M  I 

>.  NECiritNf'S  CATALOO  NUN 

Sir 

I 1.  TYRE  or  Niro  NT  1 NIMOO  COVCNIO  | 

j MS  Thesis 

«.  riNroNMiNd  ono.  mnort  numncn 

1.  CONTRACT  ON  ORANT  NUNl 

ftww 

10.  r NOON  AM  ELEMENT.  nnojj 
ARIA  0 WORK  UNIT  NUMEE 

CT.  TASK 
AS 

It.  RINORT  OATS 

December,  1976 

•1.  numoin  or  r aou 

103 

II.  SECURITY  CLASS.  ( at  dila  report! 

UNCLASSIFIED 

)lq.  O^C^ASSIFICATION/OOWH 

ORAOINO 

[ IS.  SUFFL  EMENT  ANY  NOTH 


pproved, 


alio  release;  IAW  APR  190-17 


Je^jjil  F.  Guess , Captain,  USAF 
Director  of  Information 

It.  KlY  WONO*  (CufkiM,  an  firm,  aid » II  nacaaaary  and  Ida nitty  »r  Mock  wwljfj 

scientist/Engineer  productivity 

INDIVIDUAL/ORGANIZATIONAL  VARIABLES 


20.  ktntACT  JIBBS  an  rararaa  atda  It  naeaaa oar  «4  Idantlty  »r  Mack  tSBSSS  / 

scientist/engineer  productivity  in  a selected  Air  Force  RAD 
laboratory  was  measured  with  the  number  of  eight  different  types  of 
output  as  reported  by  135  scientists/engineers.  A weighting  scheme 
was  developed  from  the  results  of  a questionnaire  administered  to 
53  first  level  supervisors  in  the  laboratory.  The  correlation  be- 
tween the  raw  sum  of  the  eight  output  types  and  the  weighted  sum 
was,  r= .985.  23  predictor  variables  were  compared  to  the  reported 

output  and  seven  were  shown  to  be  consistently  associated  with  the  ^ 


OO,: 


innoN  or  1 nov  ••  n omoutc 


UNCLASSIFIED 

••CONI TV  CLASSIFICATION  Of  THIS  RAPE  < 


“5 


reported  productivity  measures.  Education  xevex,  rrequency 
of  attendance  at  professional  society  meetings,  communication  in 
and  outside  a scientist/engineer's  section,  participation  in  set- 
ting work  goals  and  perceived  rewards  for  quantity  of  output  were 
positively  associated  with  productivity.  Length  of  time  under 
the  same  supervisor  was  negatively  associated  with  productivity. 
Canonical  correlation  analysis  provided  support  for  the  existence 
of  broad  based  types  of  output. 


UNCLASSIFIED 


